Some evidence on the accuracy of forecasts in R & D projects

In recent years many formal attempts, e.g. Baker & Pound, (1964), have been made to help R & D management assess the worth of R & D projects and select those most worthy of inclusion in the R & D programme. Most of these attempts are based on financial appraisals of a project's worth and require estimates of the costs and likely revenues that will result from the adoption of a given project. It is clear, therefore, that the validity of any measure of a worth for a project is dependent upon the accuracy of the estimates of cost and revenue associated with that project. Unfortunately the limited amount of evidence so far presented suggests, in particular, that cost estimates are extremely inaccurate.
In this paper, therefore, we try to add some further evidence on forecast accuracy for a number of industrial R & D projects. We begin with a short review of the existing literature evidence and continue with an analysis of the past effectiveness of R & D work and, in particular, of forecast accuracy.     

A NEW APPROACH TO DEPRECIATION MEASUREMENT FOR PUBLIC UTILITIES: COMMENT

This paper examines the use of a simple heuristic for evaluating projects. We posit that ranking projects by IRR and rejecting marginal projects can be superior to a NPV rule if 1) project managers have incentives to overstate cash flow forecasts that occur late in a project's life, 2) project rankings determine project acceptance because not all positive NPV project's are accepted, and 3) a project's IRR is greater than the WACC. In these instances, the IRR heuristic undervalues distant cash flows and thus, reduces project managers' incentives to positively bias forecasts.     

The accuracy of project cost and duration estimates in industrial R&D

Abstract . This study of the accuracy of cost and duration estimates, both initial and intermediate, for industrial R&D projects, is based mainly on analysis of the records of 475 projects in four varying research organizations. Mean ratios of actual to estimated cost ranging from 0.97 to 1.51 are obtained, and mean ratios of actual to estimated duration from 1.39 to 3.04. The results are generally similar to those of other British and American studies, with which they are compared.
There is no evidence that the information gained as projects progress enables their future cost and duration to be estimated more accurately; at best, the accuracy of such estimates remains constant. No effect of project size on estimate accuracy is found; effects of project length on accuracy, and of time (i.e. increasing experience) on accuracy are found only in one organization each. The pattern of expenditure over time is examined, and found on average to be not far from linear, though with wide individual project variations.
When the individual project ratios are reduced by constant factors representing optimistic or pessimistic bias (derived from the mean ratios, and assumed to be characteristic of the firm), and then subjected to a log transformation to make their distribution more symmetrical, the remaining variation, which is a measure of the inaccuracy of estimation for the individual projects, is closely similar in the four organizations, and not greatly different in the organizations covered by other studies when the data are similarly treated.     

Bias and Error in Mine Project Capital Cost Estimation

Previous analyses of small samples of mining projects have found that feasibility studies tend to underestimate the as-built capital costs of the project. Our review of 63 international mining and smelting projects confirms that as-built capital costs are, on average, 14% higher than as estimated in the bankable feasibility study. There is little attenuation over time of this bias in capital cost estimation, appearing to reflect an absence of learning on the part of the project sponsor or the consulting engineering firm. We argue that this persistence of bias is instead intentional and rational, driven by a scarcity of project financing and the need by project sponsors to inflate the project economics in a bid to secure financing. We find some empirical support for our contention. A second phase of the analysis examines estimation error. Roughly half of all projects' as-built capital costs fall outside of the expected ± 15% of the feasibility study capital cost estimate, even after allowing for intentional estimation bias. Cost overruns of 100% or more happen in roughly 1 out of 13 projects. Smaller projects have less estimation accuracy than large projects. Finally, our analysis of the cost overrun data reveals that a shifted lognormal probability distribution should be used when modeling mining project capital costs in a Monte Carlo analysis.     

可再生能源政府支持成本估算解析

提高政府支持成本估算的准确率,可为各国对未来新能源需求、供应、贸易以及投资决策提供更加准确的数据支持。《世界能源展望2010》对2007—2009年的可再生能源发电和生物燃料的政府支持成本进行了估算,并对2010-2035年的政府支持成本进行了预测。基于此,对《世界能源展望2010》中可再生能源政府支持成本的估算方法进行了介绍和评价,以期完善估算方法,为提高政府支持成本估算的准确度提供参考。     

Sales forecast errors for new product projects

It is difficult to avoid errors in sales forecasts for new industrial products, especially when the forecasts are made at the new product screening stage. But if managers are aware that a forecast is likely to be in serious error, they can take steps to deal with the uncertainty. A research project analyzed 185 new product projects to develop four major indicators of sales forecast error. The indicators can be derived for specific firms from typical project screening questions and can be used to alert new product managers to special development problems that may be present.     

THE CUTOFF BENEFIT-COST RATIO SHOULD EXCEED ONE

The engineering economics textbook rule of accepting all projects with a benefit-cost ratio in excess of one worth fails to consider the bias introduced from there being typically more poor projects than good projects. Bayesian statistics provides an analytical solution to the problem of combining prior information with an engineer's estimates. With all distributions normal, the posterior mean (to be used for decision making) becomes a weighted average of the prior mean, and the mean derived from the engineer's estimates, A simple formula provides the optimal cut-off benefit-cost ratio. This value typically exceeds the textbook's unity cut-off benefit-cost ratio.     

The learning curve, R & D management, and marketing strategies for new products

This paper represents a review of the learning phenomenon which is useful in evaluating the potential of new industrial products. The learning curve in quantitative form provides a means for forecasting costs and prices in many industrial situations. While there are many metho-dological problems to be solved in any specific new project evaluation, the utilization of the learning curve should lead to improved estimates of costs, revenues, and profits. The important role of R & D management in connection with the learning curve is discussed. The interrelationships involving R & D, the learning curve, and marketing strategies are examined.     

Housing Adjustment Costs: Their Impact on Mobility and Housing Demand Elasticities

The role of housing adjustment costs in influencing mobility decisions, and the relationship between these and housing demands is not well understood. Here we try to examine how changes in adjustment costs as well as in such demand factors as income, influence the decision of households to adjust their housing consumption through moving. Empirically, we use a sample of recently moving homeowners from the PSID and provide evidence indicating adjustment costs are more important than other demand factors in influencing mobility decisions. We also argue that sample selection may bias estimates of housing demand elasticities when the sample is restricted to recent movers because the influence of adjustment costs in the mobility decision are not accounted for. Testing for this using the Heckman two-stage procedure, however, showed this was not a significant problem.     

Subjective Performance Assessment of Innovation Projects*

Performance assessment of innovation projects is a central issue in innovation management research. Using existing literature, a model is developed to assess the performance of new product and new service development projects. In this model, project performance is defined as a combination of a formatively indicated operational performance construct and a reflectively indicated product performance construct. The validity of this model is tested based on a sample of 219 innovation projects assessed by innovation managers. Using only the innovation managers' responses, it is, however, not possible to distinguish between operational and product performance. The impact of common method bias and informant bias is subsequently assessed using a subsample of 128 of these 219 innovation projects that are assessed by the innovation manager and the project leader. These latter results show that operational and product performance are two distinct constructs. In addition, the multitrait–multimethod analyses show that especially the more abstract items of performance, such as the perceptions of quality, captured knowledge, competitive advantage, gained reputation, and customer satisfaction, suffer from random error and informant bias. Project leaders appear to be better informed to assess operational performance, while innovation managers are better in assessing product performance. The paper concludes with a qualitative comparison of several alternative performance models: the project performance model as derived from the literature, a similar (misspecified) reflective performance model, two stand‐alone models in which operational and product performance are assessed separately, and a mixed model that uses a combination of innovation managers' and project managers' data. Based on this comparison, it is advised to use either the stand‐alone models for operational performance and product performance or the mixed model whereby the project leader assesses operational performance and the innovation manager the product performance of an innovation project.     

Impacts of sustainability certification on farm income: Evidence from small-scale specialty green tea farmers in Vietnam

Sustainability certification for small-scale farms has gained considerable momentum in developing countries. However, evidence on the economic benefits of certification schemes in the context of domestically consumed foods and beverages is scarce. This paper addresses this gap by evaluating the impacts of sustainability standards on the selling prices, sales, costs and net income of small-scale specialty green tea farming in Vietnam. We analyze original survey data on 401 smallholder green tea farmers and employ propensity score matching to mitigate self-selection bias. Our estimates reveal that certified green tea producers are able to obtain a higher average selling price and sales value. In addition, although certification leads to a dramatic increase in the costs of hired labor, net farm income is still significantly improved.     

基于时间序列分析的输变电工程变压器价格短期预测研究

变压器价格是影响输变电工程投资的重要因素,但是由于影响变压器价格因素众多,因此对变压器价格进行科学预测有一定的难度,而输变电工程投资管控又迫切需要对变压器价格进行准确地预测。鉴于此,文章以2010~2018年输变电工程变压器价格为研究样本,在描述变压器价格变动轨迹的基础上,分别以移动平均法、单指数平滑法、双指数平滑法、非周期的Holt-Winter法、AR模型和MA模型对变压器价格进行预测,给出2019年和2020年变压器价格的预测值并进行优劣判断。研究结果表明:双指数平滑法预测精度相对较高,而引入多种时间序列模型进行变压器价格预测,能够给出投资决策者多种选择,可以提高决策的精确性和柔性,对输变电工程实现精准投资和良性投资控制具有一定的积极意义。     

The management of Army R & D projects

This study of the accuracy of cost and duration estimates for British defence R & D projects is based on an analysis of the records of 29 Army projects, initiated since 1960. The results are generally similar to American defence studies with which they are compared. The reasons for schedule slippages and cost escalations are examined and it is suggested that performance could be improved by more thorough project feasibility and definition studies and by a stronger project management organization.     

Estimating the Causal Effect of Entry on Generic Drug Prices Using Hatch–Waxman Exclusivity

Competition among generics helps keep drug prices low and control medical costs. Good estimates of the effect on price of the entry of another generic competitor would inform competition policy and test oligopoly theories. However, identifying the causal effect of entry is difficult since the number of firms that compete in a market is endogenously determined. We exploit provisions of the 1984 Hatch–Waxman Act to identify a causal effect. We find that ignoring endogenous selection into generic drug markets imparts a significant downward bias to the estimates of the effects of two and three competitors on generic drug prices.     

Ex post evaluation of PPP government-led renegotiations: Impacts on the financing of road infrastructure

Renegotiations are often claimed to be the major pitfall in public–private partnership projects. In fact, the literature and empirical evidence suggest a bias toward favoring concessionaires, although there are well-known examples of harmful unilateral decisions by governments. This article establishes a distinction between structural renegotiations (with changes in the risk-sharing agreement) and nonstructural renegotiations (without shifting risk) and analyzes the implications of structural renegotiations in a road concession by comparing the ex ante and ex post financial implications for government, concessionaire, and users. The selected project is one that started as a shadow toll and was later changed to real toll. The analysis shows that the renegotiation essentially transferred costs to the users. The financial burden for the government decreased and the internal rate of return (IRR) of the concessionaire increased slightly although the overall risk profile of the concession decreased. The users supported a 40% higher cost, and the system gained an additional cost component (the electronic tolling system). Overall the global financial costs increased 40% when compared with the initial regime. The article also sets out some discussions and implications for policymakers.     

Price indices for industrial R&D: A project level approach

In this paper a fairty tractable approach which managers could use to infer the rate of increase in relative RBD costs over time using actual RBD expenditure data is demonstrated. Two common measures of relative cost increase which in a sense represent bounds, the Laspeyres and Faaschs indices, are estimated for a firm, based on representative projects performed by the firm. The results which are consistent with rnothear study using company level data indicate that the official R&D statistics tend to exaggwate the increase during 1M9-79 in industrial RhQ performance. In particular using the GNP deftstor, one would appuently Qwmimate real 1979 R&D expenditures (in 1963 dolfatrs) by almost 20% in the chemical industry atone.     

Estimate accuracy and causes of delay in an engineering research laboratory

Some of the main results of a study of the accuracy of cost and duration estimates in this laboratory have already been published. This short article adds some further information about the accuracy of duration estimates for stages of projects (as opposed to whole projects), and about the reasons for upward revisions of duration estimates.     

the evaluation of innovation projects

Abstract . Any realistic assessment of research projects must allow for possible failures. Scaling down the Potential gain from each project by its probability of achievement ignores the expenditure on unsuccessful projects and is therefore misleading. The method proposed here is more sound: it uses a concept termed the Survival Probability Function, which is simply the probability that a Project will still exist at various times in the future. This function can be derived from estimates for the survival factors and durations of project stages. The function can then be used to calculate the statistics of project successes and failures, and can be employed in a standard Risk Analysis. More simply, if can be used as a weighting function for cash flow patterns. This leads to a comparison method for projects with different risks and different potential returns, and thus to project portfolio selection.     

Pipeline management for the acquisition of industrial projects

The management and controlling of the industrial project acquisition process, with its staged, funnel-like structure, has been neglected in empirical research so far—even though an effective acquisition process is crucial for economic success. In addition to profitability, the main goal of industrial companies is to implement a pipeline process that ensures that the pipeline is always filled with acquisition projects and results in a constant stream of orders over time. The projects considered are characterized by complexity, uniqueness, and discontinuity. Based on a large-scale survey among German industrial companies, we describe the industrial project acquisition process, where the projects mentioned above are sold on a non-continuous basis to industrial customers. We assess how the industrial project acquisition process is really structured, whether a sales funnel management is applicable and utilized by sales management to monitor and control this process, and whether a sophisticated sales funnel management contributes to enhanced economic success.     

Examining the heterogeneity in calorie–income elasticities: A meta-analysis

Several studies conducted over the past three decades on calorie–income relationship appear to give inconclusive results. This paper reviews these studies and employs meta-regression analysis to examine the potential publication bias in the calorie–income elasticity, as well as the impact of specific study attributes on the elasticities that have been reported in the empirical literature. A total of 40 empirical studies, which yielded 99 estimated elasticities were considered. The results indicate the presence of publication bias in the reported elasticities. Besides, the estimates revealed evidence of positive and significant empirical effect of income on calorie intake from all the studies that goes beyond publication bias. Study attributes such as ranking of the journal, panel data used in the analysis, whether expenditure was used as proxy for income, year of primary survey, sample size, and numbers of the years of primary data were found to have statistically significant impacts on the reported calorie–income elasticities in the literature.