Predicting binary outcomes

We address the issue of using a set of covariates to categorize or predict a binary outcome. This is a common problem in many disciplines including economics. In the context of a prespecified utility (or cost) function we examine the construction of forecasts suggesting an extension of the  and  maximum score approach. We provide analytical properties of the method and compare it to more common approaches such as forecasts or classifications based on conditional probability models. Large gains over existing methods can be attained when models are misspecified.     

Tests of Predictive Ability for Vector Autoregressions Used for Conditional Forecasting

Many forecasts are conditional in nature. For example, a number of central banks routinely report forecasts conditional on particular paths of policy instruments. Even though conditional forecasting is common, there has been little work on methods for evaluating conditional forecasts. This paper provides analytical, Monte Carlo and empirical evidence on tests of predictive ability for conditional forecasts from estimated models. In the empirical analysis, we examine conditional forecasts obtained with a VAR in the variables included in the DSGE model of Smets and Wouters (American Economic Review 2007; 97 : 586–606). Throughout the analysis, we focus on tests of bias, efficiency and equal accuracy applied to conditional forecasts from VAR models. Copyright © 2016 John Wiley & Sons, Ltd.     

Efficient forecast tests for conditional policy forecasts

Central Banks regularly make forecasts, such as the Fed’s Greenbook forecast, that are conditioned on hypothetical paths for the policy interest rate. While there are good public policy reasons to evaluate the quality of such forecasts, up until now, the most common approach has been to ignore their conditional nature and apply standard forecast efficiency tests. In this paper we derive tests for the efficiency of conditional forecasts. Intuitively, these tests involve implicit estimates of the degree to which the conditioning path is counterfactual and the magnitude of the policy feedback over the forecast horizon. We apply the tests to the Greenbook forecast and the Bank of England’s inflation report forecast, finding some evidence of forecast inefficiency. Nonetheless, we argue that the conditional nature of the forecasts made by central banks represents a substantial impediment to the analysis of their quality—stronger assumptions are needed and forecast inefficiency may go undetected for longer than would be the case if central banks were instead to report unconditional forecasts.     

Financial nowcasts and their usefulness in macroeconomic forecasting

Financial data often contain information that is helpful for macroeconomic forecasting, while multi-step forecast accuracy benefits from incorporating good nowcasts of macroeconomic variables. This paper considers the usefulness of financial nowcasts for making conditional forecasts of macroeconomic variables with quarterly Bayesian vector autoregressions (BVARs). When nowcasting quarterly financial variables’ values, we find that taking the average of the available daily data and a daily random walk forecast to complete the quarter typically outperforms other nowcasting approaches. Using real-time data, we find gains in out-of-sample forecast accuracy from the inclusion of financial nowcasts relative to unconditional forecasts, with further gains from the incorporation of nowcasts of macroeconomic variables. Conditional forecasts from quarterly BVARs augmented with financial nowcasts rival the forecast accuracy of mixed-frequency dynamic factor models and mixed-data sampling (MIDAS) models.     

Multi-step forecasting in emerging economies: An investigation of the South African GDP

To forecast at several, say h, periods into the future, a modeller faces a choice between iterating one-step-ahead forecasts (the IMS technique), or directly modeling the relationship between observations separated by an h-period interval and using it for forecasting (DMS forecasting). It is known that structural breaks, unit-root non-stationarity and residual autocorrelation may improve DMS accuracy in finite samples, all of which occur when modelling the South African GDP over the period 1965–2000. This paper analyzes the forecasting properties of 779 multivariate and univariate models that combine different techniques of robust forecasting. We find strong evidence supporting the use of DMS and intercept correction, and attribute their superior forecasting performance to their robustness in the presence of breaks.     

Efficient information theoretic inference for conditional moment restrictions

The generalised method of moments estimator may be substantially biased in finite samples, especially so when there are large numbers of unconditional moment conditions. This paper develops a class of first-order equivalent semi-parametric efficient estimators and tests for conditional moment restrictions models based on a local or kernel-weighted version of the Cressie–Read power divergence family of discrepancies. This approach is similar in spirit to the empirical likelihood methods of Kitamura et al. [2004. Empirical likelihood-based inference in conditional moment restrictions models. Econometrica 72, 1667–1714] and Tripathi and Kitamura [2003. Testing conditional moment restrictions. Annals of Statistics 31, 2059–2095]. These efficient local methods avoid the necessity of explicit estimation of the conditional Jacobian and variance matrices of the conditional moment restrictions and provide empirical conditional probabilities for the observations.     

Normal mixture GARCH(1,1): applications to exchange rate modelling

Some recent specifications for GARCH error processes explicitly assume a conditional variance that is generated by a mixture of normal components, albeit with some parameter restrictions. This paper analyses the general normal mixture GARCH(1,1) model which can capture time variation in both conditional skewness and kurtosis. A main focus of the paper is to provide evidence that, for modelling exchange rates, generalized two‐component normal mixture GARCH(1,1) models perform better than those with three or more components, and better than symmetric and skewed Student's t‐GARCH models. In addition to the extensive empirical results based on simulation and on historical data on three US dollar foreign exchange rates (British pound, euro and Japanese yen), we derive: expressions for the conditional and unconditional moments of all models; parameter conditions to ensure that the second and fourth conditional and unconditional moments are positive and finite; and analytic derivatives for the maximum likelihood estimation of the model parameters and standard errors of the estimates. Copyright © 2006 John Wiley & Sons, Ltd.     

Replicating the Results in ‘A New Model of Trend Inflation’ Using Particle Markov Chain Monte Carlo

An article by Chan et al. ( 2013 ) published in the Journal of Business and Economic Statistics introduces a new model for trend inflation. They allow the trend inflation to evolve according to a bounded random walk. In order to draw the latent states from their respective conditional posteriors, they use accept–reject Metropolis–Hastings procedures. We reproduce their results using particle Markov chain Monte Carlo (PMCMC), which approaches drawing the latent states from a different technical point of view by relying on combining Markov chain Monte Carlo and sequential Monte Carlo methods. To conclude: we are able to reproduce the results of Chan et al. ( 2013 ). Copyright © 2015 John Wiley & Sons, Ltd.     

Unconditional tests for association in 2 × 2 contingency tables in the total sum fixed design

The asymptotic approach and Fisher's exact approach have often been used for testing the association between two dichotomous variables. The asymptotic approach may be appropriate to use in large samples but is often criticized for being associated with unacceptable high actual type I error rates for small to medium sample sizes. Fisher's exact approach suffers from conservative type I error rates and low power. For these reasons, a number of exact unconditional approaches have been proposed, which have been seen to be generally more powerful than exact conditional counterparts. We consider the traditional unconditional approach based on maximization and compare it to our presented approach, which is based on estimation and maximization. We extend the unconditional approach based on estimation and maximization to designs with the total sum fixed. The procedures based on the Pearson chi‐square, Yates's corrected, and likelihood ratio test statistics are evaluated with regard to actual type I error rates and powers. A real example is used to illustrate the various testing procedures. The unconditional approach based on estimation and maximization performs well, having an actual level much closer to the nominal level. The Pearson chi‐square and likelihood ratio test statistics work well with this efficient unconditional approach. This approach is generally more powerful than the other p‐value calculation methods in the scenarios considered.     

The COVID-19 shock and challenges for inflation modelling

We document the impact of COVID-19 on inflation modelling within a vector autoregression (VAR) model and provide guidance for forecasting euro area inflation during the pandemic. We show that estimated parameters are strongly affected, leading to different and sometimes implausible projections. As a solution, we propose to augment the VAR by allowing the residuals to have a fat-tailed distribution instead of a Gaussian one. This also outperforms with respect to unconditional forecasts. Yet, what brings sizeable forecast gains during the pandemic is adding meaningful off-model information, such as that entailed in the Survey of Professional Forecasters. The fat-tailed VAR loses part, but not all of its relative advantage compared to the Gaussian version when producing conditional inflation forecasts in a real-time setup. It is the joint fat-tailed errors and multi-equation modelling that manage to robustify models against extreme observations; in a single-equation model the same solution is less effective.     

The performance of alternative forecasting methods for SETAR models

We compare a number of methods that have been proposed in the literature for obtaining h-step ahead minimum mean square error forecasts for self-exciting threshold autoregressive (SETAR) models. These forecasts are compared to those from an AR model. The comparison of forecasting methods is made using Monte Carlo simulation. The Monte-Carlo method of calculating SETAR forecasts is generally at least as good as that of the other methods we consider. An exception is when the disturbances in the SETAR model come from a highly asymmetric distribution, when a Bootstrap method is to be preferred.An empirical application calculates multi-period forecasts from a SETAR model of US gross national product using a number of the forecasting methods. We find that whether there are improvements in forecast performance relative to a linear AR model depends on the historical epoch we select, and whether forecasts are evaluated conditional on the regime the process was in at the time the forecast was made.     

Variable selection, estimation and inference for multi-period forecasting problems

This paper conducts a broad-based comparison of iterated and direct multi-period forecasting approaches applied to both univariate and multivariate models in the form of parsimonious factor-augmented vector autoregressions. To account for serial correlation in the residuals of the multi-period direct forecasting models we propose a new SURE-based estimation method and modified Akaike information criteria for model selection. Empirical analysis of the 170 variables studied by Marcellino, Stock and Watson (2006) shows that information in factors helps improve forecasting performance for most types of economic variables although it can also lead to larger biases. It also shows that SURE estimation and finite-sample modifications to the Akaike information criterion can improve the performance of the direct multi-period forecasts.     

Downside risk management and VaR-based optimal portfolios for precious metals,oil and stocks

Value-at-Risk (VaR) is used to analyze the market downside risk associated with investments in six key individual assets including four precious metals, oil and the S&P 500 index, and three diversified portfolios. Using combinations of these assets, three optimal portfolios and their efficient frontiers within a VaR framework are constructed and the returns and downside risks for these portfolios are also analyzed. One-day-ahead VaR forecasts are computed with nine risk models including calibrated RiskMetrics, asymmetric GARCH type models, the filtered Historical Simulation approach, methodologies from statistics of extremes and a risk management strategy involving combinations of models. These risk models are evaluated and compared based on the unconditional coverage, independence and conditional coverage criteria. The economic importance of the results is also highlighted by assessing the daily capital charges under the Basel Accord rule. The best approaches for estimating the VaR for the individual assets under study and for the three VaR-based optimal portfolios and efficient frontiers are discussed. The VaR-based performance measure ranks the most diversified optimal portfolio (Portfolio #2) as the most efficient and the pure precious metals (Portfolio #1) as the least efficient.     

Exploiting tail shape biases to discriminate between stable and student t alternatives

The nonnormal stable laws and Student t distributions are used to model the unconditional distribution of financial asset returns, as both models display heavy tails. The relevance of the two models is subject to debate because empirical estimates of the tail shape conditional on either model give conflicting signals. This stems from opposing bias terms. We exploit the biases to discriminate between the two distributions. A sign estimator for the second‐order scale parameter strengthens our results. Tail estimates based on asset return data match the bias induced by finite‐variance unconditional Student t data and the generalized autoregressive conditional heteroscedasticity process.     

Comparison of intraday probabilistic forecasting of solar irradiance using only endogenous data

Accurate solar forecasts are necessary to improve the integration of solar renewables into the energy grid. In recent years, numerous methods have been developed for predicting the solar irradiance or the output of solar renewables. By definition, a forecast is uncertain. Thus, the models developed predict the mean and the associated uncertainty. Comparisons are therefore necessary and useful for assessing the skill and accuracy of these new methods in the field of solar energy.The aim of this paper is to present a comparison of various models that provide probabilistic forecasts of the solar irradiance within a very strict framework. Indeed, we consider focusing on intraday forecasts, with lead times ranging from 1 to 6 h. The models selected use only endogenous inputs for generating the forecasts. In other words, the only inputs of the models are the past solar irradiance data. In this context, the most common way of generating the forecasts is to combine point forecasting methods with probabilistic approaches in order to provide prediction intervals for the solar irradiance forecasts. For this task, we selected from the literature three point forecasting models (recursive autoregressive and moving average (ARMA), coupled autoregressive and dynamical system (CARDS), and neural network (NN)), and seven methods for assessing the distribution of their error (linear model in quantile regression (LMQR), weighted quantile regression (WQR), quantile regression neural network (QRNN), recursive generalized autoregressive conditional heteroskedasticity (GARCHrls), sieve bootstrap (SB), quantile regression forest (QRF), and gradient boosting decision trees (GBDT)), leading to a comparison of 20 combinations of models.None of the model combinations clearly outperform the others; nevertheless, some trends emerge from the comparison. First, the use of the clear sky index ensures the accuracy of the forecasts. This derived parameter permits time series to be deseasonalized with missing data, and is also a good explanatory variable of the distribution of the forecasting errors. Second, regardless of the point forecasting method used, linear models in quantile regression, weighted quantile regression and gradient boosting decision trees are able to forecast the prediction intervals accurately.     

Can the random walk model be beaten in out-of-sample density forecasts? Evidence from intraday foreign exchange rates

It has been documented that random walk outperforms most economic structural and time series models in out-of-sample forecasts of the conditional mean dynamics of exchange rates. In this paper, we study whether random walk has similar dominance in out-of-sample forecasts of the conditional probability density of exchange rates given that the probability density forecasts are often needed in many applications in economics and finance. We first develop a nonparametric portmanteau test for optimal density forecasts of univariate time series models in an out-of-sample setting and provide simulation evidence on its finite sample performance. Then we conduct a comprehensive empirical analysis on the out-of-sample performances of a wide variety of nonlinear time series models in forecasting the intraday probability densities of two major exchange rates—Euro/Dollar and Yen/Dollar. It is found that some sophisticated time series models that capture time-varying higher order conditional moments, such as Markov regime-switching models, have better density forecasts for exchange rates than random walk or modified random walk with GARCH and Student-t innovations. This finding dramatically differs from that on mean forecasts and suggests that sophisticated time series models could be useful in out-of-sample applications involving the probability density.     

Forecasting risk measures using intraday and overnight information

Volatility forecasts are important for a number of practical financial decisions, such as those related to risk management. When working with high-frequency data from markets that operate during a reduced time, an approach to deal with the overnight return volatility is needed. In this context, we use heterogeneous autoregressions (HAR) to model the variation associated with the intraday activity, with distinct realized measures as regressors, and, to model the overnight returns, we use augmented GARCH type models. Then, we combine the HAR and GARCH models to generate forecasts for the total daily return volatility. In an empirical study, for returns on six international stock indices, we analyze the separate modeling approach in terms of its out-of-sample forecasting performance of daily volatility, Value-at-Risk and Expected Shortfall relative to standard models from the literature. In particular, the overall results are favorable for the separate modeling approach in comparison with some HAR models based on realized variance measures for the whole day and the standard GARCH model.     

Decay factor optimisation in time weighted simulation — Evaluating VaR performance

We propose an optimisation approach for determining the optimal decay factor in time weighted (BRW) simulation. The backtesting of the BRW simulation, which involves different decay factors, together with a broad range of competing VaR models, has been performed on a sample of seven stock indexes and two commodities: gold and WTI oil. The results obtained show that the BRW simulation with an optimised decay factor relative to the Lopez (1998) size-adjusted function is among the best performing VaR models, second only to the conditional extreme value approach (McNeil & Frey, 2000). The optimised decay factors are sufficiently stable over time, giving economic justification to the optimisation because they do not change over longer time periods. Unlike most of the VaR models tested, in the large majority of cases, the optimised BRW model passes the Basel II criteria but yields significantly lower VaR forecasts than the extreme value approaches, thus resulting in a lower idle capital, i.e. lower costs.     

Conditional nonparametric frontier models for convex and nonconvex technologies: a unifying approach

The explanation of productivity differentials is very important to identify the economic conditions that create inefficiency and to improve managerial performance. In the literature two main approaches have been developed: one-stage approaches and two-stage approaches. Daraio and Simar (2005, J Prod Anal 24(1):93–121) propose a fully nonparametric methodology based on conditional FDH and conditional order-m frontiers without any convexity assumption on the technology. However, convexity has always been assumed in mainstream production theory and general equilibrium. In addition, in many empirical applications, the convexity assumption can be reasonable and sometimes natural. Lead by these considerations, in this paper we propose a unifying approach to introduce external-environmental variables in nonparametric frontier models for convex and nonconvex technologies. Extending earlier contributions by Daraio and Simar (2005, J Prod Anal 24(1):93–121) as well as Cazals et al. (2002, J Econometrics 106:1–25), we introduce a conditional DEA estimator, i.e., an estimator of production frontier of DEA type conditioned to some external-environmental variables which are neither inputs nor outputs under the control of the producer. A robust version of this conditional estimator is proposed too. These various measures of efficiency provide also indicators of convexity which we illustrate using simulated and real data. Cinzia Daraio received Research support from the Italian Ministry of Education Research on Innovation Systems Project (iRis) “The reorganization of the public system of research for the technological transfer: governance, tools and interventions” and from the Italian Ministry of Educational Research Project (MIUR 40% 2004) “System spillovers on the competitiveness of Italian economy: quantitative analysis for sectoral policies” which are acknowledged. Léopold Simar received Research support from the “Interuniversity Attraction Pole”, Phase V (No. P5/24) from the Belgian Government (Belgian Science Policy) is acknowledged.     

High quantiles estimation with Quasi-PORT and DPOT: An application to value-at-risk for financial variables

Recurrent ⿿black swans⿿ financial events are a major concern for both investors and regulators because of the extreme price changes they cause, despite their very low probability of occurrence. In this paper, we use unconditional and conditional methods, such as the recently proposed high quantile (HQ) extreme value theory (EVT) models of DPOT (Duration-based Peak Over Threshold) and quasi-PORT (peaks over random threshold), to estimate the Value-at-Risk with very small probability values for an adequately long and major financial time series to obtain a reasonable number of violations for backtesting. We also compare these models and other alternative strategies through an out-of-sample accuracy investigation to determine their relative performance within the HQ context. Policy implications relevant to estimation of risk for extreme events are also provided.