证券咨询机构预测能力分析--以新股定位为例

本文以四家证券咨询机构对108只新股的开盘价预测为研究对象,文中首先给出数据和研究方法;接着对各机构预测误差进行比较以检验不同机构的预测能力;紧接着分析了机构预测价对新股短期投资的参考价值。     

Evaluation of the best M4 competition methods for small area population forecasting

The ‘M4’ forecasting competition results were featured recently in a special issue of the International Journal of Forecasting and included projections for demographic time series. We sought to investigate whether the best M4 methods could improve the accuracy of small area population forecasts, which generally suffer from much higher forecast errors than regions with larger populations. The aim of this study was to apply the top ten M4 forecasting methods to produce 5- and 10-year forecasts of small area total populations using historical datasets from Australia and New Zealand. Forecasts were compared against the actual population numbers and forecasts from two simple benchmark models. The M4 methods were found to perform relatively well compared to our benchmarks. In the light of these findings, we discuss possible future directions for small area population forecasting research.     

Exploring the representativeness of the M5 competition data

The main objective of the M5 competition, which focused on forecasting the hierarchical unit sales of Walmart, was to evaluate the accuracy and uncertainty of forecasting methods in the field to identify best practices and highlight their practical implications. However, can the findings of the M5 competition be generalized and exploited by retail firms to better support their decisions and operation? This depends on the extent to which M5 data is sufficiently similar to unit sales data of retailers operating in different regions selling different product types and considering different marketing strategies. To answer this question, we analyze the characteristics of the M5 time series and compare them with those of two grocery retailers, namely Corporación Favorita and a major Greek supermarket chain, using feature spaces. Our results suggest only minor discrepancies between the examined data sets, supporting the representativeness of the M5 data.     

The M4 Competition: Results,findings, conclusion and way forward

The M4 competition is the continuation of three previous competitions started more than 45 years ago whose purpose was to learn how to improve forecasting accuracy, and how such learning can be applied to advance the theory and practice of forecasting. The purpose of M4 was to replicate the results of the previous ones and extend them into three directions: First significantly increase the number of series, second include Machine Learning (ML) forecasting methods, and third evaluate both point forecasts and prediction intervals. The five major findings of the M4 Competitions are: 1. Out Of the 17 most accurate methods, 12 were “combinations” of mostly statistical approaches. 2. The biggest surprise was a “hybrid” approach that utilized both statistical and ML features. This method’s average sMAPE was close to 10% more accurate than the combination benchmark used to compare the submitted methods. 3. The second most accurate method was a combination of seven statistical methods and one ML one, with the weights for the averaging being calculated by a ML algorithm that was trained to minimize the forecasting. 4. The two most accurate methods also achieved an amazing success in specifying the 95% prediction intervals correctly. 5. The six pure ML methods performed poorly, with none of them being more accurate than the combination benchmark and only one being more accurate than Naïve2. This paper presents some initial results of M4, its major findings and a logical conclusion. Finally, it outlines what the authors consider to be the way forward for the field of forecasting.     

Fathoming empirical forecasting competitions’ winners

The M5 forecasting competition has provided strong empirical evidence that machine learning methods can outperform statistical methods: in essence, complex methods can be more accurate than simple ones. Regardless, this result challenges the flagship empirical result that led the forecasting discipline for the last four decades: keep methods sophisticatedly simple. Nevertheless, this was a first, and we can argue that this will not happen again. There has been a different winner in each forecasting competition. This inevitably raises the question: can a method win more than once (and should it be expected to)? Furthermore, we argue for the need to elaborate on the perks of competing methods, and what makes them winners?     

A modified Teunter-Syntetos-Babai method for intermittent demand forecasting

Intermittent demand refers to the specific demand pattern with frequent periods of zero demand. It occurs in a variety of industries including industrial equipment, automotive and specialty chemicals. In some industries or some sectors of industry, even majority of products are in intermittent demand pattern. Due to the usually small and highly variable demand sizes, accurate forecasting of intermittent demand has always been challenging.However, accurate forecasting of intermittent demand is critical to the effective inventory management. In this study we present a band new method - modified TSB method for the forecasting of intermittent demand. The proposed method is based on TSB method, and adopts similar strategy, which has been used in mSBA method to update demand interval and demand occurrence probability when current demand is zero. To evaluate the proposed method, 16289 daily demand records from the M5 data set that are identified as intermittent demands according to two criteria, and an empirical data set consisting three years’ monthly demand history of 1718 medicine products are used. The proposed mTSB method achieves the best results on MASE and RMASE among all comparison methods on the M5 data set. On the empirical data set, the study shows that mTSB attains an ME of 0.07, which is the best among six comparison methods. Additionally, on the MSE measurement, mTSB shows a similar result as SES, both of which outperform other methods.     

Predicting/hypothesizing the findings of the M5 competition

The scientific method consists of making hypotheses or predictions and then carrying out experiments to test them once the actual results have become available, in order to learn from both successes and mistakes. This approach was followed in the M4 competition with positive results and has been repeated in the M5, with its organizers submitting their ten predictions/hypotheses about its expected results five days before its launch. The present paper presents these predictions/hypotheses and evaluates their realization according to the actual findings of the competition. The results indicate that well-established practices, like combining forecasts, exploiting explanatory variables, and capturing seasonality and special days, remain critical for enhancing forecasting performance, re-confirming also that relatively new approaches, like cross-learning algorithms and machine learning methods, display great potential. Yet, we show that simple, local statistical methods may still be competitive for forecasting high granularity data and estimating the tails of the uncertainty distribution, thus motivating future research in the field of retail sales forecasting.     

Differencing versus nondifferencing in factor-based forecasting

This paper studies performance of factor-based forecasts using differenced and nondifferenced data. Approximate variances of forecasting errors from the two forecasts are derived and compared. It is reported that the forecast using nondifferenced data tends to be more accurate than that using differenced data. This paper conducts simulations to compare root mean squared forecasting errors of the two competing forecasts. Simulation results indicate that forecasting using nondifferenced data performs better. The advantage of using nondifferenced data is more pronounced when a forecasting horizon is long and the number of factors is large. This paper applies the two competing forecasting methods to 68 I(1) monthly US macroeconomic variables across a range of forecasting horizons and sampling periods. We also provide detailed forecasting analysis on US inflation and industrial production. We find that forecasts using nondifferenced data tend to outperform those using differenced data.     

What determines forecasters’ forecasting errors?

This paper contributes to the growing body of literature in macroeconomics and finance on expectation formation and information processing by analyzing the relationship between expectation formation at the individual level and the prediction of macroeconomic aggregates. Using information from business tendency surveys, we present a new approach of analyzing forecasters’ qualitative forecasting errors. Based on a quantal response approach with misclassification, we define forecasters’ qualitative mispredictions in terms of deviations from the qualitative rational expectation forecast, and relate them to the individual and macro factors that are driving these mispredictions. Our approach permits a detailed analysis of individual forecasting decisions, allowing for the introduction of individual and economy-wide determinants that affect the individual forecasting error process.     

The value added by machine learning approaches in forecasting

This discussion reflects on the results of the M4 forecasting competition, and in particular, the impact of machine learning (ML) methods. Unlike the M3, which included only one ML method (an automatic artificial neural network that performed poorly), M4’s 49 participants included eight that used either pure ML approaches, or ML in conjunction with statistical methods. The six pure (or combination of pure) ML methods again fared poorly, with all of them falling below the Comb benchmark that combined three simple time series methods. However, utilizing ML either in combination with statistical methods (and for selecting weightings) or in a hybrid model with exponential smoothing not only exceeded the benchmark, but performed at the top. While these promising results by no means prove ML to be a panacea, they do challenge the notion that complex methods do not add value to the forecasting process.     

Forecasting Errors: the Track Record

This Briefing Paper is the last of a series of three about forecasting. In this one we examine our forecasting record; it complements the February paper in which we analysed the properties of our forecasting model in terms of the error bands attached to the central forecast.
There are many ways of measuring forecasting errors, and in the first part of this Briefing Paper we describe briefing how we have tackled the problem. (A more detailed analysis can be found in the Appendix.) In Part II we report and comment upon the errors in our forecasts of annual growth rates and show how our forecasting performance has improved over the years. In Part III we focus on quarterly forecasts up to 8 quarters ahead, and compare our forecasting errors with measurement errors in the oficial statistics; with the estimation errors built into our forecast equations; and with the stochastic model errors we reported last February. A brief summary of the main conclusions is given below.     

Robust recurrent network model for intermittent time-series forecasting

This paper describes a deep-learning-based time-series forecasting method that was ranked third in the accuracy challenge of the M5 competition. We solved the problem using a deep-learning approach based on DeepAR, which is an auto-regressive recurrent network model conditioned on historical inputs. To address the intermittent and irregular characteristics of sales demand, we modified the training procedure of DeepAR; instead of using actual values for the historical inputs, our model uses values sampled from a trained distribution and feeds them to the network as past values. We obtained the final result using an ensemble of multiple models to make a robust and stable prediction. To appropriately select a model for the ensemble, each model was evaluated using the average weighted root mean squared scaled error, calculated for all levels of a wide range of past periods.     

Correlated daily time series and forecasting in the M4 competition

We participated in the M4 competition for time series forecasting and here describe our methods for forecasting daily time series. We used an ensemble of five statistical forecasting methods and a method that we refer to as the correlator. Our retrospective analysis using the ground truth values published by the M4 organisers after the competition demonstrates that the correlator was responsible for most of our gains over the naïve constant forecasting method. We identify data leakage as one reason for its success, due partly to test data selected from different time intervals, and partly to quality issues with the original time series. We suggest that future forecasting competitions should provide actual dates for the time series so that some of these leakages could be avoided by participants.     

The performance of the global bottom-up approach in the M5 accuracy competition: A robustness check

The M5 accuracy competition has presented a large-scale hierarchical forecasting problem in a realistic grocery retail setting in order to evaluate an extended range of forecasting methods, particularly those adopting machine learning. The top ranking solutions adopted a global bottom-up approach, by which is meant using global forecasting methods to generate bottom level forecasts in the hierarchy and then using a bottom-up strategy to obtain coherent forecasts for aggregate levels. However, whether the observed superior performance of the global bottom-up approach is robust over various test periods or only an accidental result, is an important question for retail forecasting researchers and practitioners. We conduct experiments to explore the robustness of the global bottom-up approach, and make comments on the efforts made by the top-ranking teams to improve the core approach. We find that the top-ranking global bottom-up approaches lack robustness across time periods in the M5 data. This inconsistent performance makes the M5 final rankings somewhat of a lottery. In future forecasting competitions, we suggest the use of multiple rolling test sets to evaluate the forecasting performance in order to reward robustly performing forecasting methods, a much needed characteristic in any application.     

Applicability of the M5 to Forecasting at Walmart

The M5 Forecasting Competition, the fifth in the series of forecasting competitions organized by Professor Spyros Makridakis and the Makridakis Open Forecasting Center at the University of Nicosia, was an extremely successful event. This competition focused on both the accuracy and uncertainty of forecasts and leveraged actual historical sales data provided by Walmart. This has led to the M5 being a unique competition that closely parallels the difficulties and challenges associated with industrial applications of forecasting. Like its precursor the M4, many interesting ideas came from the results of the M5 competition which will continue to push forecasting in new directions.In this article we discuss four topics around the practitioners view of the application of the competition and its results to the actual problems we face. First, we examine the data provided and how it relates to common difficulties practitioners must overcome. Secondly, we review the relevance of the accuracy and uncertainty metrics associated with the competition. Third, we discuss the leading solutions and their implications to forecasting at a company like Walmart. We then close with thoughts about a future M6 competition and further enhancements that can be explored.     

Forecasting the M4 competition weekly data: Forecast Pro’s winning approach

Forecast Pro forecasted the weekly series in the M4 competition more accurately than all other entrants. Our approach was to follow the same forecasting process that we recommend to our users. This approach involves determining the Key Performance Metric (KPI), establishing baseline forecasts using our automated expert selection algorithm, reviewing those baseline forecasts and customizing forecasts where needed. This article explores why this approach worked well for weekly data, discusses the applicability of the M4 competition to business forecasting and proposes some potential improvements for future competitions to make them more relevant to business forecasting.     

An ensemble approach to GEFCom2017 probabilistic load forecasting

We present an ensembling approach to medium-term probabilistic load forecasting which ranked second out of 73 competitors in the defined data track of the GEFCom2017 qualifying match. In addition to being accurate, the ensemble method is highly scalable, due to the fact that it had to be applied to nine quantiles in ten zones and for six rounds. Candidate forecasts were generated using random settings for input data, covariates, and learning algorithms. The best candidate forecasts were averaged to create the final forecast, with the number of candidate forecasts being chosen based on their accuracy in similar validation periods.     

Czech female managers and their wages

This paper examines gender gaps in employment and wages among top- and lower-level managerial employees in the Czech Republic at the time of its accession to the EU. Using both least-squares and matching-based decomposition techniques, we find the wage gap among comparable men and women to be sizeable, but quite similar across firm hierarchy levels. The key reason why the average relative wage position of female top managers is worse compared to lower-ranking female employees is that women tend not to be at the helm of the highest-paying companies. Overall, the representation of women at the top of Czech firms as well as the structure of the gender wage gap there appears quite similar to those in the US.     

The M4 Competition: 100,000 time series and 61 forecasting methods

The M4 Competition follows on from the three previous M competitions, the purpose of which was to learn from empirical evidence both how to improve the forecasting accuracy and how such learning could be used to advance the theory and practice of forecasting. The aim of M4 was to replicate and extend the three previous competitions by: (a) significantly increasing the number of series, (b) expanding the number of forecasting methods, and (c) including prediction intervals in the evaluation process as well as point forecasts. This paper covers all aspects of M4 in detail, including its organization and running, the presentation of its results, the top-performing methods overall and by categories, its major findings and their implications, and the computational requirements of the various methods. Finally, it summarizes its main conclusions and states the expectation that its series will become a testing ground for the evaluation of new methods and the improvement of the practice of forecasting, while also suggesting some ways forward for the field.     

Investigating the accuracy of cross-learning time series forecasting methods

The M4 competition identified innovative forecasting methods, advancing the theory and practice of forecasting. One of the most promising innovations of M4 was the utilization of cross-learning approaches that allow models to learn from multiple series how to accurately predict individual ones. In this paper, we investigate the potential of cross-learning by developing various neural network models that adopt such an approach, and we compare their accuracy to that of traditional models that are trained in a series-by-series fashion. Our empirical evaluation, which is based on the M4 monthly data, confirms that cross-learning is a promising alternative to traditional forecasting, at least when appropriate strategies for extracting information from large, diverse time series data sets are considered. Ways of combining traditional with cross-learning methods are also examined in order to initiate further research in the field.