Development and application of a new methodology of evaluating industrial R&D projects

This paper proposes a new methodology of evaluating industrial R&D projects to assess the effectiveness of future R&D in terms of financial credibility, to prioritize them efficiently by clear criteria to reduce the time and burden consumed by both project leaders and management staffs. A new methodology has been developed and applied to all Sumitomo Electric Industries (SEI) R&D projects, and is recognized as a useful system for evaluating many R&D projects in a short period, such as 150 projects per month.     

A portfolio–evaluation framework for selecting R&D projects

This study aims to form the basis for constructing a framework for evaluating alternative portfolios of R&D projects. This study provides an extensive literature review on portfolio selection. Most of the existing studies deal with the portfolio selection problem by evaluating individual projects and then seeking ways to combine them for an R&D portfolio. However, the combination of individually good projects unnecessarily constitutes the optimal portfolio. In particular, this study discusses three portfolio effects: (1) the difference between the preference for the portfolio as a whole and the preference for the projects, (2) the interrelation among projects, (3) the size of portfolio selection problems. This study develops a three–phase framework for evaluating R&D portfolios and proposes a new taxonomy of the portfolio attributes (i.e. independent, interrelated, and synergistic). This study concludes with a discussion of future research, directed toward increasing the applicability of portfolio–selection approaches for managing R&D portfolios.     

Budgets for research and development: an empirical study of 69 Swedish firms

It was found that, as a rule, firms decide upon individual projects and do not specify in advance a fixed level for the R & D budget. The decision about individual R & D projects and/or the R & D budget was most frequently made by the president of the company. The implication of our results for various kinds of research on the economics of R & D are discussed. Questionnaires were sent out to 124 large Swedish firms, asking how they determined budgets for research and development (R & D) and who decided about the size of these budgets. 94 firms answered and 69 of these undertook R & D.     

Budgets for research and development: an empirical study of 69 Swedish firms

Questionnaires were sent out to 124 large Swedish firms, asking how they determined budgets for research and development (R & D) and who decided about the size of these budgets. 94 firms answered and 69 of these undertook R & D.
It was found that, as a rule, firms decide upon individual projects and do not specify in advance a fixed level for the R & D budget. The decision about individual R & D projects and/or the R & D budget was most frequently made by the president of the company.
The implication of our results for various kinds of research on the economics of R & D are discussed.     

R & D and the company's requirements

Numerous models have been put forward in the literature for the selection and evaluation of projects, many of which are based on optimizing the return on investment by the company in R & D projects. This is a noble aim but in practice the financial data available are often of dubious accuracy and so many other factors are involved in actually adopting a portfolio of selected projects that project selection often becomes a theoretical exercise. It is suggested that the selection of projects is in essence the adoption of a strategy which is within the policy and long-term aims of the company and also within the policy and scientific development of the R & D part of the company. Such a strategy is usually sub-optimum for both the company and the R & D division, as it is a compromise between a desire for high pay-back investments by the company and the maintenance of a scientifically balanced R & D involving some long-term, unknown return, investments. A method is currently being experimented with by Unilever Research which illustrates where conflicts arise between the policy of the main company and that of research, and assists management in arriving at a strategy which most closely fulfils the needs of both policies. Both the concept of the method and its practical use are described in this paper.     

Assessing collaborative, pre-competitive R&D projects: the case of the UK LINK scheme

The paper addresses the evaluation of outcomes of collaborative, pre-competitive R&D projects. It draws on some empirical analysis carried out on data and information gathered under the UK 'LINK' scheme, a programme supporting R&D collaborative projects and assessing collaboration outcomes. A new indicator of performance, based on the outputs of LINK projects, has been constructed. It provides a relative and quite consistent measure of performance for making comparisons among different LINK projects. However it does not correlate with the more subjective grade applied by LINK's own management. For further improving evaluating mechanisms, more attention should be paid to the benefits that universities and companies as well as governments are drawing from R&D collaborations.     

The quality of subjective probabilities of technical success in R & D

Employing various measures for the quality of subjective probabilities of technical success in R & D a detailed analysis of 4 data sets taken from different R & D organizations shows that the probability assessments roughly specify the populations to which R & D projects belong, but are extremely unreliable indicators of the eventual outcome of individual activities. Unintentional errors and conscious biases are identified as accounting for the unsatisfactory quality of subjective probabilities in R & D. Finally, some measures are suggested which might improve the quality of the probability assessments.     

Some evidence on the accuracy of forecasts in R & D projects

In recent years many formal attempts, e.g. Baker & Pound, (1964), have been made to help R & D management assess the worth of R & D projects and select those most worthy of inclusion in the R & D programme. Most of these attempts are based on financial appraisals of a project's worth and require estimates of the costs and likely revenues that will result from the adoption of a given project. It is clear, therefore, that the validity of any measure of a worth for a project is dependent upon the accuracy of the estimates of cost and revenue associated with that project. Unfortunately the limited amount of evidence so far presented suggests, in particular, that cost estimates are extremely inaccurate.
In this paper, therefore, we try to add some further evidence on forecast accuracy for a number of industrial R & D projects. We begin with a short review of the existing literature evidence and continue with an analysis of the past effectiveness of R & D work and, in particular, of forecast accuracy.     

Project selection in a small university R&D laboratory

Abstract: The case study presented in this paper deals with the problem of selecting interrelated R & D projects in a small university R & D laboratory under constrained resources. A constrained optimization model was used, applying a multiattribute utility approach.     

The value of a ‘failed’ R&D project: an emerging evaluation framework for building innovative capabilities1

In literature and R&D organizations alike, project success consists in minimizing the deviations from set targets in terms of quality, cost and time. The main management task is to execute and monitor progress to reduce risks – assuming that project attributes are known, necessary resources can be estimated and a reasonable time table can be agreed upon. In such a context, evaluating project success is easy. However, in an innovative context, setting project targets initially is difficult and the contributions of the projects sometimes are of an unexpected nature. This paper investigates if projects can be evaluated in terms of how they contribute to the building of innovative capabilities of the firm instead of independently. Based on a case study at the Régie Autonome des Transports Parisians and the theoretical framework of innovation fields, a framework for evaluating projects from an additional perspective is proposed. Based on the following four criteria: financial resources, the development of a structured, refined and expanded strategic vision, developed competences (with related suppliers) and identification of knowledge gaps (occasionally with related partners for knowledge production), this framework shows how seemingly failed R&D projects can instead be considered as invaluable to the overall innovation process.     

The management of Army R & D projects

This study of the accuracy of cost and duration estimates for British defence R & D projects is based on an analysis of the records of 29 Army projects, initiated since 1960. The results are generally similar to American defence studies with which they are compared. The reasons for schedule slippages and cost escalations are examined and it is suggested that performance could be improved by more thorough project feasibility and definition studies and by a stronger project management organization.     

The Debiasing of Forecasts in Research and Development

Abstract . A possible way to improve the accuracy of forecasts is to estimate (and subsequently remove) any systematic bias by reference to the bias found in similar forecasts in the past. This paper discusses the applicability to industrial R & D of a method proposed by Summers for debiasing estimates of the costs of military projects by means of a regression equation derived from past experience, and gives the results of applying a similar method to estimates of costs, sales, product prices, and return factors for ten industrial development projects.     

The Directions for Technological Change: Alternative Economic Majorities and Opportunity Costs

Governments redirect R & D resources. Thegovernment supports R & D in technologies and in partsof the country that the private sector is unlikely tosupport, and the government provides funds forinexperienced businesses and for minority-ownedbusinesses when the private sector will not. Becausethe distribution of income and wealth is so unequal,our conventional methods of evaluating the socialusefulness of the redirection of the resources may notwork well. Redirection of R & D resources to meetSchumpeterian and Jeffersonian objectives may actuallybe efficient despite generating less economic surplusgiven the current distribution of income and wealth.     

Controlling R & D projects by networks

Abstract . This paper discusses the use of network analysis in the management of a large scale engineering project carried out within the R & D department of an engineering company. It describes the use of precedence diagrams and concludes that these are simpler to use than the more commonly encountered PERT nomenclature. This method has now been extended to cover other projects in the R & D department and it is concluded that their successful introduction has been encouraged by the interest and support of top management.     

R&D projects in the science sector

The purpose of this research was to determine and analyse diversity in various features of R&D projects implemented in the science sector in Poland, financed mostly from public funds. More exactly, the following aspects of R&D projects implemented in the science sector in Poland were examined: the relation between the actually achieved and the initially set goal, the project management methodologies and project time management methods used, features of the project team, the relation between the plan and the actual project realisation and management difficulties encountered in practice. These aspects and their diversity were examined in the context of the whole country, of various fields of science and various research units implementing pure research (the ‘R’ area) or applied research and experimental development (the ‘D’ area). The research was conducted by means of survey methodology covering a sample of participants of R&D projects in the science sector in Poland. The results were analysed by means of statistical methods (which constitutes a clear novelty with respect to the existing literature on R&D projects) and statistically significant phenomena gave rise to several important conclusions on how R&D projects implemented in the science sector are managed and what their realisation looks like in various fields of science and various institutions, in a relatively new EU members state and a former communist country like Poland. These conclusions may be the basis for important hypotheses (which require further research) on the project management quality in the science sector in Poland and in similar countries, which is closely linked to the problem of public money allocation, its efficient spending and control. Also, practical suggestions on improvement measures in the science sector with respect to R&D projects, limited not only to countries like Poland, but of a more general nature, are formulated.     

Qualitative aspects of the evaluation and control of research and development projects

Decisions in research and development are usually based on a variety of data inputs. Some of these will be concerned with the returns offered by projects in money terms. Others will relate to more qualitative aspects of research work such as, perhaps, the establishment of a base in a new area of technology or the achievement of a stronger market position. This paper describes and illustrates how checklists may be used to assemble qualitative data for purposes of decision. These were formulated in an industrial R & D laboratory and the illustrative examples in the text make reference to real projects. The checklists described are an integral part of a comprehensive system of project evaluation and control in R & D which includes an analysis of the financial implications using net present value or discounted cash flow rate of return of the project risk, analysis and sensitivity tests. The detail of this part of the scheme is conventional and is therefore omitted from the paper.     

Literature indexing systems for corporate R & D strategy: a case-study in the pharmaceutical industry

Abstract . Information retrieval systems in R & D laboratories are normally used to provide information about the state of the art in fields of corporate interest and to help to solve day-to-day problems. The authors show that such systems can also be used as a tool for planning by R & D managers, corporate planners and others involved in evaluating and setting strategic objectives for the firm. Their approach is to use the Science Citation Index to ascertain the specific institutional setting and geographical locations of researchers in whom material published by the subject firm excited a 'resonance'.
In their study they used the publications lists covering a five-year period provided by two West German companies. They point out that it is possible not only to assess the scientific standing of the R & D laboratory but also to get early warning of spin-offs into hitherto unrecognized fields, to identify 'markets' for the profitable exchange of information and to locate complementary sources of expertise.     

Steady state and portfolio concept in R & D management

This paper describes a new approach for closing the gap between the tools for strategic goal management and operational management in Pharmaceutical R&D. The strategic level may concern with expectation of sales, rising from new products and with total R & D budget. The operational level concerns with project selection, project and multiproject management problems, solution of project related technical problems, determination of the size of the various functions and the functional management. This new method combines the R & D project portfolio management with a steady state concept allowing the attainment of a constant number of projects under development and a continuous full use of the capacity available.     

The learning curve, R & D management, and marketing strategies for new products

This paper represents a review of the learning phenomenon which is useful in evaluating the potential of new industrial products. The learning curve in quantitative form provides a means for forecasting costs and prices in many industrial situations. While there are many metho-dological problems to be solved in any specific new project evaluation, the utilization of the learning curve should lead to improved estimates of costs, revenues, and profits. The important role of R & D management in connection with the learning curve is discussed. The interrelationships involving R & D, the learning curve, and marketing strategies are examined.     

Using mean-Gini and stochastic dominance to choose project portfolios with parameter uncertainty

Although a variety of models have been studied for project portfolio selection, many organizations still struggle to choose a potentially diverse range of projects while ensuring the most beneficial results. The use of the mean-Gini framework and stochastic dominance to select portfolios of research and development (R&D) projects has been gaining attention in the literature despite the fact that such approaches do not consider uncertainty regarding the projects’ parameters. This article discusses, with relation to project portfolio selection through a mean-Gini approach and stochastic dominance, the impact of uncertainty on project parameters. In the process, Monte Carlo simulation is considered in evaluating the impact of parametric uncertainty on project selection. The results show that the influence of uncertainty is significant enough to mislead managers. A more robust selection policy using the mean-Gini approach and Monte Carlo simulation is proposed.