A comparative analysis of R&D project evaluation methods

A large number of methodologies for Research and Development (R&D) project evaluation have been developed and reported in the literature over the last few decades. Almost all of this literature focused on the mechanisms and underlying theoretical foundation on which the evaluation methods are based. Very little attention has been paid to the effectiveness and suitability of the techniques, especially a comparison of the techniques. This paper presents a comparative study on a number of classes of R&D evaluation methods based on the Analytic Hierarchy Process. We look into the various factors and characteristics of R&D evaluation methods that affect their suitability. A sensitivity analysis was also carried out to determine the critical factors.     

R&D project selection: Productivity considerations

The paper consists of a theoretical discussion of the circumstances justifying the use of R&D to increase the productivity of a firm's manufacturing capability. There are three possibilities: Increase the productivity of existing capacity; re-equip with purchased existing best-practice technology; or use R&D to develop new technology.
The choice depends on which of the four following situations exist: 1) When the productivity of existing capacity is well below its potential and is also well below that of the industry's best-practice; in this case the right strategy is first to bring actual productivity up to potential, and then to consider the purchase of best-practice. 2) When productivity is well up to potential but is well below best practice; the advice is then to purchase best-practice. 3) When existing capacity is already best-practice but productivity is well below capacity; in this case productivity of existing plant should be increased. 4) When existing capacity is bestpractice and productivity is near the maximum; then and only then is R&D to develop new technology justified.
The paper concludes with a brief account of the process to be undergone between the establishment of a case for R&D using the above criteria and, if R&D is undertaken and completed, the retrospective review of its outcome.     

Strategic evaluation and selection of R&D projects

The most important families of R&D project evaluation and selection methodologies and associated techniques are briefly reviewed. For each family, generally one or two methods were chosen to be analysed from the point of view of the integration with the strategy. But, first some definitions and some aspects of the new strategic role of the evaluation and selection of R&D projects, which were in mind during the interviews, are presented.
Finally, some empirical results of the use of these methodologies in France and Japan are presented. The strong and weak points of the most prevalent methods are described as they are considered by the users.     

R&D project selection and scheduling for organizations facing product obsolescence

Research and development is very crucial in an organization that faces rapid product obsolescence. Our thesis is that R&D project scheduling and resource planning should be born out of the optimum product launch plan which, in turn, should be in adherence to the growth rate targeted by the organization. Such an approach would ensure that the R&D activities of the organization are in tune with the Organization's goals. The absence of such coordinated R&D planning would hamper the productivity and profitability of the organization. This paper attempts to develop a methodology and model that would enable the streamlining of R&D project schedule. A real life case application has also been illustrated.     

R&D project selection for promoting the efficiency of energy use

Screening criteria for selecting energy R& D projects are discussed and applied to a dozen projects currently underway. The impact on energy balance and environmental quality cannot be seen as the only, nor even the dominating, attribute. In the cases studied, the R&D portfolio seeks to achieve a balance between these factors and the other criteria related to technology and the market. In the event the development work is carried out at an independent R&D centre, it will be necessary to have a committed industrial partner from an early stage onwards in order to ensure objectivity, market perception and provide the best chance of commercialisation.     

Aspects of project control in R&D

This paper examines the concept of project control in the context of an R&D laboratory, and reviews some of the methods of presenting control or monitoring information that have been found of use by management for different types of work. The concepts described are those which the author has found of practical value in his own work, and which also figure in some R&D Management training courses within British Gas.     

Stakeholder analysis for R&D project management

R&D projects affect the interests of different stakeholders in different ways. Understanding the stakeholders and analysing their interests helps in the better management of R&D projects. In this paper we discuss a methodology to systematically analyse the stakeholders of R&D projects. This methodology includes Freeman's (1984) three levels of analysis: rational, process and transactional. Based on these three levels, the stakeholder management capability of an R&D project is determined. The final stage is based on Mitchell et al . (1997) approach to analysing the dynamics of stakeholders. This methodology is illustrated using a New Zealand case relating to a road pricing R&D project.     

Uncertainty and stress in an R&D project environment

Abstract
R&D project managers work in an environment in which the uncertainty and complexity of engineering development problems are an important source of episodic job stress. A study of the US. Navy's top major project R&D managers (N = 118) was conducted to test the use of various coping skills and social support as preventive stress management techniques for this population. It was found that perceived environmental uncertainty (PEU) had a direct, positive effect on psychological distress of these project managers while social support had a therapeutic effect on their experience of burnout. While problem focussed coping had a small, positive effect on burnout, neither coping skills nor social support were found to have a primary prevention effect upon perceived environmental uncertainty (PEU).
Engineering managers engaged in high technology R&D are confronted by a wide range of uncertainties, risks and changing demands that give rise to the stress response with its attendant physiological and psychological changes (Asterita, 1985). Adams (1980) has argued that an episodic versus chronic dichotomy of stressors is valuable because it distinguishes between intermittent stressors of finite duration (episodic) and ongoing stressors of indeterminant duration (chronic). This article reports the results of a study of uncertainty as an episodic work stressor for the U.S. Navy's high technology R&D project managers. The results suggest that R&D project managers should develop social support systems and structures to maintain well-being and avert distress and strain.     

Social capital and creativity in R&D project teams

The existing research contributes to our understanding about the value of social capital in a wide range of social science disciplines; however, it does not well address the role of social capital in creativity for research and development (R&D) project teams in a given context. Using a sample of 54 R&D project teams in high-technology firms of Taiwan, we examined the impacts of social capital on creativity of R&D project teams from an intra-team perspective. Results of factor analysis revealed four factors extracted from the concept of social capital, namely social interaction, network ties, mutual trust, and shared goals. Findings suggested that social interaction and network ties had significant and positive impacts on creativity of R&D project teams, but mutual trust and shared goals did not. Managerial implications for managing social capital in R&D project teams are discussed.     

Monitoring R&D project costs against pre-specified targets

The monitoring and interpretation of costs incurred during a project against pre-specified targets is the most common task which R&D managers at all levels have to undertake. Graphical interpretation of such cost and target figures can provide useful feedback as to financial performance.
Spreadsheets provide an ideal environment for the manipulation of figures and production of graphs and, although not widely used, the programming facilities incorporated in such packages also allow for menu driven systems to be developed which can be operated without a knowledge of the spreadsheet's command structure thus greatly widening their area of application.
This article describes some of the elements of such a system and details its ability to automatically produce four specified graphs taken from accountancy, statistical process control and forecasting which have been shown to offer a valuable insight into cost control generally and, it is proposed, within R&D projects in particular.     

Project selection in Post Office telecommunications R&D

In an industry in which the benefits of R & D are often difficult to quantify, numerate processes for the selection of projects and portfolios are of limited usefulness. The processes that are used provide formal means, necessary in a large organization, to enable managerial judgment at an appropriate level to be applied to both project selection and the balance of the portfolios.     

Organizational learning through post–project reviews in R&D

Post–project reviews are one opportunity to systematically improve performance in subsequent projects. However, a survey reveals that only one out of five R&D projects receives a post–project review. Post–project reviews – if they take place – are typically constrained by lack of time and attention as well as lack of personal interest and ability. They focus mostly on technical output and bureaucratic measurements; process–related factors such as project management are rarely discussed.
In this paper we review the role of post–project meetings as a tool to improve organizational learning at the group level. Based on 27 in–depth interviews with R&D managers carried out between 1997 and 2001, we categorize four classes of learning impediments. These difficulties are not easily resolved, as is illustrated by examples from Hewlett–Packard, DaimlerChrysler, SAP, Unisys, the US Army, and others. We propose a five–level post–project review capability maturity model, identifying some of the key capabilities that need to be in place in order to advance to the next process maturity level. Most companies reside on the first or second maturity level. Our conclusion is that many companies give away great potential for competence building by neglecting post–project reviews as a tool for systematic inter–project learning.     

International R&D project evaluation by multinational corporations in the electronics and IT industry of Singapore

This paper explores a number of variables associated with the evaluation of international R&D projects by multinational corporations (MNCs) in the electronics and IT industry of Singapore. Empirical analysis of the data collected from R&D managerial executives suggests a series of considerations in relation to their R&D investments. It is desirable if R&D can create a potential impact on the growth of their regional and international businesses. In considering the risks associated with an R&D project, a balance approach would be appropriate when demanding a return on investment. In particular, the consistence with customer demands, the achievement of time‐based competitiveness, the training of R&D manpower and the development of conducive innovation environments are fundamental to the success of international R&D projects.     

Project selection in a small university R&D laboratory

Abstract: The case study presented in this paper deals with the problem of selecting interrelated R & D projects in a small university R & D laboratory under constrained resources. A constrained optimization model was used, applying a multiattribute utility approach.     

Institutional means-ends decoupling work in industrial R&D project implementation

Managing goals is a key network management function and is critical in the implementation of industrial R&D projects. In this paper, we explore the implementation of an industrial R&D project, focusing in particular upon the role of means-ends decoupling work to understand how the goals are managed. We combine several data sources in our case research to explore project implementation through an understanding of means-ends decoupling work. We collected in-depth interviews, archival records and field observations within the R&D research setting of an industrial R&D project in the period of 2015 to 2017. Our findings identify three types of means-ends decoupling work in R&D project implementation: ‘work on’ causal complexity, ‘work at’ behavioural invisibility, and ‘work with’ practice multiplicity. In addition, we uncover six dynamic micro-mechanisms that collectively influence the making and nature of means-ends decoupling work and therefore serve to allow for the fluid switching of work as the institutional conditions permit. Overall, our findings have significant implications for understanding means-ends decoupling as a highly skilled network competence for managing R&D project implementation goals.     

Empowering project scope decisions: introducing R&D content graphs

There have been several studies confirming the strategic and operational importance of choosing a proper project scope. There is a lack of empirically tested managerial tools to assist in this matter, and studies with longitudinally analysed R&D projects with respect to content are scarce. This article discusses the management of advanced engineering in the automotive industry. The study presents a structured approach where individual project leaders together with top management could prepare project material, discuss, visualize, and evaluate the content around a common tool. This tool, i.e. the R&D content graphs, facilitates project content selection, both during the start‐up and during the execution phase of the project. The studied project has been studied for nearly two years using an abductive case‐based research design. Since the project has been longitudinally studied, it has also been possible to test the graphs' predicting capabilities. The graphs are based on a visualization of the project content along five identified dimensions. The results reveal that the tools bring structure to project scope discussions, have some predictive value, and can also function as a vertical as well as a horizontal communication tool.     

Price indices for industrial R&D: A project level approach

In this paper a fairty tractable approach which managers could use to infer the rate of increase in relative RBD costs over time using actual RBD expenditure data is demonstrated. Two common measures of relative cost increase which in a sense represent bounds, the Laspeyres and Faaschs indices, are estimated for a firm, based on representative projects performed by the firm. The results which are consistent with rnothear study using company level data indicate that the official R&D statistics tend to exaggwate the increase during 1M9-79 in industrial RhQ performance. In particular using the GNP deftstor, one would appuently Qwmimate real 1979 R&D expenditures (in 1963 dolfatrs) by almost 20% in the chemical industry atone.     

A portfolio–evaluation framework for selecting R&D projects

This study aims to form the basis for constructing a framework for evaluating alternative portfolios of R&D projects. This study provides an extensive literature review on portfolio selection. Most of the existing studies deal with the portfolio selection problem by evaluating individual projects and then seeking ways to combine them for an R&D portfolio. However, the combination of individually good projects unnecessarily constitutes the optimal portfolio. In particular, this study discusses three portfolio effects: (1) the difference between the preference for the portfolio as a whole and the preference for the projects, (2) the interrelation among projects, (3) the size of portfolio selection problems. This study develops a three–phase framework for evaluating R&D portfolios and proposes a new taxonomy of the portfolio attributes (i.e. independent, interrelated, and synergistic). This study concludes with a discussion of future research, directed toward increasing the applicability of portfolio–selection approaches for managing R&D portfolios.