Methods for analyzing data from Delphi panels: Some evidence from a forecasting study

Delphi and other methods of using expert opinion to generate forecasts can be a useful tool for planning, impact assessment, and policy analysis. Unfortunately, little is known about the accuracy of forecasts produced using these methods, so their utility is limited at present. Based on the logic of the Delphi method, I suggest that: 1) forecast accuracy should increase across rounds of a Delphi iteration, 2) there is a positive correlation between a panelist's uncertainty about a forecast and his or her shift in forecast from round to round, 3) forecasts weighted by self-reported confidence will be more accurate than unweighted forecasts, and 4) the use of robust estimates of location as summaries of expert opinion yield better forecasts than nonrobust measures. A Delphi experiment provides little support to any of these hypotheses. This finding suggests that traditional assumptions about the proper methods for analyzing a Delphi study may be inappropriate.     

Innovation forecasting

Technological forecasting is premised on a certain orderliness of the innovation process. Myriad studies of technological substitution, diffusion, and transfer processes have yielded conceptual models of what matters for successful innovation, but most technological forecasts key on limited empirical measures quite divorced from those innovation process models. We glean a number of concepts from various innovation models, then present an array of bibliometric measures that offer the promise of operationalizing these concepts. Judicious combination of such bibliometrics with other forms of evidence offers an enriched form of technological forecasting we call “innovation forecasting.” This provides a good means to combine technological trends, mapping of technological interdependencies, and competitive intelligence to produce a viable forecast. We illustrate by assessing prospects for ceramic engine technologies.     

Volatility forecasting: combinations of realized volatility measures and forecasting models

This article examines financial time series volatility forecasting performance. Different from other studies which either focus on combining individual realized measures or combining forecasting models, we consider both. Specifically, we construct nine important individual realized measures and consider combinations including the mean, the median and the geometric means as well as an optimal combination. We also apply a simple AR(1) model, an SV model with contemporaneous dependence, an HAR model and three linear combinations of these models. Using the robust forecasting evaluation measures including RMSE and QLIKE, our empirical evidence from both equity market indices and exchange rates suggests that combinations of both volatility measures and forecasting models improve the forecast performance significantly.     

Continuous forecasting methods

The concept of a continuously operating system of expert opinion, a “collective brain” involving the combination of Delphi and network chart procedures, is discussed. The computer-aided process described by the author uses four types of information: the forecast of external actions of the environment, the values of control parameters as function of time, the current values of internal parameters, and estimates obtained from the principal experts.     

Dissecting models' forecasting performance

The fact that the predictive performance of models used in forecasting stock returns, exchange rates, and macroeconomic variables is not stable and varies over time has been widely documented in the forecasting literature. Under these circumstances excessive reliance on forecast evaluation metrics that ignores this instability in forecasting accuracy, like squared errors averaged over the whole forecast evaluation sample, masks important information regarding the temporal evolution of relative forecasting performance of competing models. In this paper we suggest an approach based on the combination of the Cumulated Sum of Squared Forecast Error Differential (CSSFED) of Welch and Goyal (2008) and the Bayesian change point analysis of Barry and Hartigan (1993) that tracks the contribution of forecast errors to the aggregate measures of forecast accuracy observation by observation. In doing so, it allows one to track the evolution of the relative forecasting performance over time. We illustrate the suggested approach by using forecasts of the GDP growth rate in Switzerland.     

Brownian agent-based technology forecasting

Today's innovation process is best characterized by nonlinearity and interaction. Agent-based models build on these concepts, but have not been useful in practice because they are either too complex or too simple to make a good match with reality. As a remedy, we employ a Brownian agent model with intermediate complexity to produce value-added technology forecasting. As an illustration with Korea's software industry data, computer simulation is carried out. Attracted by higher technology value, agents concentrate on specific technology regions, and form co-existing major technology regions of high density. A rough comparison with actual software production data exhibits a fair reflection of reality, and supports the underlying idea that economic motivation of agents should be considered.     

旋转机械设备状态监测技术在设备管理中的应用

针对设备运行中存在的问题和隐患,茨榆坨采油厂利用旋转机械设备状态监测技术,结合设备的具体结构特点和使用维修情况进行综合分析,发现设备安装和使用过程中存在的问题,并做出正确的判断,使设备能够长期正常运转,减少设备管理、维护费用,取得了较好的效果.     

On a probabilistic model of forecasting

One of the methods of studying complex objects is the construction of a mathematical model, containing such information about the object that is necessary to solve a definite problem connected with it.Mathematical modeling, based on the construction of models of various kinds can be used in forecasting. Let a forecasting object A(X) be described by vector X = (X₁, X₂,…,X_n) whose coordinates are parameters characterizing this object. The work presents a probabilistic model of forecasting and gives the example of a forecast of the object described by a set two parameters.     

Problems of forecasting and technology assessment

A broad appraisal of forecasts of U.S. national trends in population, economics, transportation, energy-use, and technology reveals some of the limitations of forecasting and some avenues for improvement. The development of greater methodological sophistication has not significantly improved forecast accuracy. The (often linear) deterioration of accuracy with lengthening of forecast time horizons proceeds regardless of method. Methodology expresses (and traces the implications of) core assumptions reflecting the forecasters' fundamental outlook. Sophisticated methodology cannot save a forecast based on faulty core assumptions.High inaccuracy results from the persistence of out-of-date core assumptions (“assumption drag”), caused by overspecialization, wishful thinking, the infrequency of forecast studies (due to the common preference for expensive approaches), and the weakness of sociopolitical forecasting. This diagnosis calls for more frequent, less elaborate, and interdisciplinary forecasting efforts.Sociopolitical forecasting, required both as a source of core assumptions for projecting other trends and to trace the social impact component of technology assessments, has suffered from a lack of specificity and meaningfulness. The greater uncertainty in forecasting technological developments requiring political decisions and large-scale programs indicates the importance of improving sociopolitical analysis. The social-indicators and scenario approaches are two means for achieving this improvement. Their potential contributions, as well as limitations, are reviewed.     

The role of forecasting in sustainability

A lack of understanding of the waste flow of durable goods complicates decision-making efforts that will increase sustainability. To address this problem, the modeling of the waste flow of durable goods is considered. Televisions are used to illustrate the requirements for forecasting and the magnitude of the associated uncertainty of the waste flow for a durable good that is impacted by technological change and/or unpredictability in field life. This example is timely due to disposal laws affecting cathode ray tubes (CRTs) and the emergence of alternate technology trajectories for televisions. The findings suggest that the reuse of lead-bearing CRT glass is worthwhile, even though flat panel TV technology will eliminate this controversial waste stream. The findings also indicate the implications of forecasting durable waste flows for decisions regarding policy, business models, required infrastructure and supply chain management.     

Determinants for forecasting technological substitution

This paper investigates the factors determining the rate of technological substitution and evaluates the prospects for forecasting models based on market-share data only. Studies on underlying causes for the substitution of one technology or product by another reveal the frequent presence of a number of factors. These factors are first discussed in general terms so as to establish their causal relationships with the rate of substitution. On the basis of the identified cause-effect relationships, a composite model is formulated that incorporates the impact of individual factors in a combination of additive and multiplicative interactions. The relative importance and sensitivity of different factors in describing the behavior of the substitution process is then studied through a system-dynamics application of the model. Finally, using three specific cases, a comparison is given between the forecasts made by the comprehensive model, incorporating factors that are known to have significant impact on the rate of substitution, with that of a generalized model for forecasting technological substitution.     

Robust estimation and inflation forecasting

This paper considers various models emerging from the Fisher effect and/or the term structure of interest rates for inflation forecasting. This paper, it is believed, makes a contribution to the literature on estimation of the models by using a procedure that is robust for non-normal errors, improving the efficiency of the estimates considerably. The Consumer Price Index series, 90 days and 180 days Australian bank-accepted bill rates, covering the sample period 1968Q1 to 1998Q4 were used in this study. Contrary to earlier findings, strong evidence was documented supporting the Fisher effect in the presence of a structural break with the break-point being at 1980Q1. The overall results suggest that the error correction model of the Fisher effect, the term structure of interest rates and short-run dynamics produce superior forecasts, in particular when the models were estimated using the robust method. These findings have important implications for economic policy analysis.     

Applications of the p-similarity method to forecasting

The p-similarity method applied previously in the prediction of pollution crises is used as a general forecasting technique.