Exploring the differences in performance measurement between research and development: evidence from a multiple case study

Researchers and practitioners have recently paid great attention to research and development (R&D) performance measurement, although it is acknowledged to be a very challenging task because of R&D intrinsic uncertainty and complexity levels. In this paper, the problem of designing a performance measurement system (PMS) for R&D activities is addressed; specifically, we investigate if and how the design of the PMS is influenced by the type of activity it is applied to, namely Basic and Applied Research or new product development (NPD). We first develop a theoretical framework that comprises the main constitutive elements of a PMS for R&D. Then the framework is used for supporting a multiple case study analysis involving eight Italian technology-intensive firms. The research results show that the criteria for designing the constitutive elements of the PMS are radically different in Basic and Applied Research and NPD. The reasons behind the observed dissimilarities in the design criteria are widely discussed in the paper, as well as their implications for R&D managers.     

Utilizing the balanced scorecard for R&D performance measurement

R&D programs are critical for many firms to achieve and sustain competitive advantage. Yet, measuring R&D performance over time can be quite complex due to inherent uncertainty. The paper responds to calls in the R&D literature to explore integrated performance measurement systems that capture financial and nonfinancial performance. We integrate the Stage-Gate approach to R&D management with the Balanced Scorecard to present a framework to show how firms can link resource commitments to these activities and the firm's strategic objectives. In this paper, we provide specific examples of how firms can apply this integrated performance measurement system to the R&D function.     

System modeling of the R&D domain through the object–process methodology: a practical tool to help R&D satisfy its customers' needs

Research and Development is an intricate process involving a host of challenging technological frontiers. This paper presents a practical tool to help R&D personnel satisfy the needs of customers by a systematic evaluation of R&D performance. To establish a sound methodology for R&D performance evaluation, we employ the object-process analysis (OPA) part of the object–process methodology. Using OPA, we gradually expose the constituents of the R&D system, its environment and the interaction between them. Through this systematic top-down refinement, we provide a comprehensive generic view of the R&D domain. The framework presented in the paper allows the user to describe, order, and inter-relate the issues that R&D managers should consider when managing, evaluating and planning R&D. As the approach addresses structure and behaviour in a unified, integrated manner, the analysis provides insights into both the static and dynamic aspects of the R&D domain and establishes a solid basis for its enterprise modeling.     

The derivation of R&D return indicators within a real options framework

The purpose of this paper is to rework the building blocks of real option applications and to introduce a basket option framework. We find that the characteristic parameters of the risk neutral density function implied in observed share prices within the real option framework represent a novel category of R&D return indicators. Empirical evidence for a set of 13 US bio-pharmaceutical companies is provided. The novel R&D return indicator can be used to analyse investor's expectations on R&D success of a particular firm. The implications of this indicator on decision making are mainly based on its information content on technological and market risk of the products under development in a particular firm. A proposal for a potential application of the stability index in innovation research is discussed as well. The study thus is at the interface between innovation research and (empirical) finance.     

Achieving contextual ambidexterity in R&D organizations: a management control system approach

Research on how managers control R&D activities has tended to focus on the performance measurement systems used to exploit existing knowledge and capabilities. This focus has been at the expense of how broader forms of management control could be used to enable R&D contextual ambidexterity, the capacity to attain appropriate levels of exploitation and exploration behaviors in the same R&D organizational unit. In this paper, we develop a conceptual framework for understanding how different types of control system, guided by different R&D strategic goals, can be used to induce and balance both exploitation and exploration. We illustrate the elements of this framework and their relations using data from biotechnology firms, and then discuss how the framework provides a basis to empirically examine a number of important control relationships and phenomena.     

Performance measurement in R&D: exploring the interplay between measurement objectives, dimensions of performance and contextual factors

Measuring research and development (R&D) performance has become a fundamental concern for R&D managers and executives in the last decades. As a result, the issue has been extensively debated in innovation and R&D management literature. The paper contributes to this growing body of knowledge, adopting a systemic and contextual perspective to look into the problem of measuring R&D performance. In particular, it explores the interplay between measurement objectives, performance dimensions and contextual factors in the design of a performance measurement system (PMS) for R&D activities. The paper relies on a multiple case study analysis that involved 15 Italian technology-intensive firms. The results indicate that firms measure R&D performance with different purposes, i.e. motivate researchers and engineers, monitor the progress of activities, evaluate the profitability of R&D projects, favour coordination and communication and stimulate organisational learning. These objectives are pursued in clusters, and the importance firms attach to each cluster is influenced by the context (type of R&D, industry belonging, size) in which measurement takes place. Furthermore, a firm's choice to measure R&D performance along a particular perspective (i.e. financial, customer, business processes or innovation and learning) is influenced by the classes of objectives (diagnostic, motivational or interactive) that are given higher priority. The implications of these results for R&D managers and scholars are discussed in the paper.     

The allocation of R&D resource

Abstract
The allocation of R&D resource in a multi-business organisation is often based more on current operating performance than on the relative potential for technological development of the businesses. The approach suggested in this paper attempts to overcome these problems by linking R&D allocation very closely to business strategy. The two key concepts are technology leverage, i.e. the extent to which technology can change the business environment in an industry to yield competitive advantage, and business leverage i.e. the current competitive strength of an individual business within an industry. Where technology leverage is weak, competitive advantage will be mainly dependent on factors other than R&D and it is suggested that levels of R&D effort at or below 1 % of turnover are appropriate. Where technology leverage is strong, R&D can be effective in maintaining strong business leverage by achieving technology mastery or in transforming a weak business position by creating a technological discontinuity. R&D intensities ranging from 3 to 15 or more % of turnover are thought to be appropriate in these cases. This framework can be used for policy debates on R&D allocation, for example whether the R&D allocation is appropriate for the business strategy or whether R&D could spearhead a more expansionist business strategy. The framework has been successfully used in ICI Chemicals and Polymers Limited during the annual R&D budget process.     

Systemic absorptive capacity: creating early-to-market returns through R&D alliances

For most complex emergent technologies, product-market success depends on efficient linkages between changing lead innovators within the R&D process. In this paper, our unit of analysis is a complex high technology product and the system of alliance linkages formed to progress a product through R&D milestones. We present a model and evidence for advancing our understanding of how achieving early-to-market returns depends on systemic absorptive capacity. This systemic absorptive capacity is the cumulative efficiency in the use of absorptive capacity to link changing lead innovators across successive milestones in R&D product development. We advance propositions of how systemic absorptive capacity can explain performance differences between rival product development systems competing for early-to-market returns with similar products through accelerating speed to market, cost and quality advantages. These explanations are contrasted with the conclusions of previous studies that have focused on absorptive capacity of single firms or single alliances in R&D.     

How to leverage the impact of R&D on product innovation? The moderating effect of management innovation

Although research and development (R&D) is a key indicator of (technological) innovation, scholars have found mixed results regarding its effect on product innovation and firm performance. In this paper, we claim that variations in R&D effectiveness can be explained by changes in a firm’s social system, in particular in its management innovation. It is still unclear how management innovation influences R&D effectiveness in terms of product innovation. In this study, we address this theoretical and empirical gap in the innovation literature. Our theoretical arguments and findings from a large-scale survey among Dutch firms show that R&D has a decreasingly positive relationship with product innovation, particularly for firms with low levels of management innovation. However, in firms with high levels of management innovation, this relationship becomes more J-shaped, especially in small and medium-sized firms. Our findings also appear to indicate that management innovation may be more important for competitive advantage than just R&D. Overall, our insights reveal that management innovation is a key moderator in explaining firms’ effectiveness in transforming R&D into successful product innovation.     

Corporate technology portfolios and R&D performance measures: a study of technology intensive firms

This paper examines the relations between technology portfolio strategies and five commonly used research and development (R&D) performance measures. Patent and financial data of 78 US-based technology companies from 1976 to 1995 were gathered and analysed to investigate how a well-managed technology portfolio can create synergy and affect R&D performance. A technology portfolio can be characterized by its composition and technology concentration. A valuable technology portfolio that consists of patents with higher average citation made and self-citation ratio can have a positive effect on firm value. Our findings suggest that large firms may enjoy advantages for technological innovation because they can exploit synergy effects of their technology portfolios. Technology concentration strategy does not work well because firms focusing on few technology fields can experience diseconomy to patents received since high-quality patents are increasingly difficult to obtain. This paper lays the groundwork for future empirical research on technology portfolio and R&D performance.     

Corporate strategy as a source of ideas

This paper looks at a number of others which bear on the interface between Corporate Strategy and R&D Planning to see if a more systematic approach can suggest a sequence to be followed so that Corporate Strategy can be seen as a source of ideas for R&D.
The purpose of R&D is defined in terms of contributing to the competitiveness of the Company or Business Unit being served. The areas of competition or threats from the external environment are then explored using a model taken from Porter's book 'Competitive Strategy'. Each area of competition is considered in turn and strategies identified which might enable the company to maintain its competitive edge. Sub-objectives for the R&D function can then be determined allocating resources and time for their completion.
It is the performance of the R&D function inj fulfilling these sub-objectives which can be the basis of measuring the effectiveness of R&D.     

Design principles for the development of measurement systems for research and development processes

Based on a comprehensive literature review and the activities of numerous case study companies, it is argued in this paper that performance measurement in R&D is a fundamental aspect to quality in R&D and to overall business performance. However, it is apparent from the case companies that many companies still struggle with the issue of R&D performance measurement. Excuses for not measuring are easily found, but there are also empirical examples and literature available with suggestions how it can be done. In this article this literature is reviewed and placed within the context of general performance control and contingency theory. Furthermore, the main measurement system design parameters are discussed and some basic system requirements are described as well as several design principles that can be useful for those who accept the challenge of establishing a meaningful measurement system.     

Research and Development in Algeria

Since 1971, that is nine years after gaining its independence, Algeria has among some Third World countries formulated an ambitious Science and Technology (S&T) Policy. In the last few years, the relevant administrative structure has been completed by installing a centralised body at the highest level. Policy itself has changed very little, and remained in some aspects well behind developments in the structure. In this paper, the author intends to consider the evolution of both structure and policy of Algeria's research and development (R&D); and the extent of their coherence or harmony and their linkages with industry.
Even though structure and policy are the key factors affecting R&D effectiveness of a country, no significant success could be achieved without strengthening the relevant network allowing for the technical competence to be used effectively. Hence, the main theme of the paper concentrates on Algeria's need to review its R&D system to allow both structure and policy to be effective or cause relevant activities to be integrated within the production system, build better technological capabilities, and work for a more efficient industrial development.     

Scale and performance in publicly funded collaborative research and development

This paper reports the results of an extensive empirical investigation on the role of project scale (reflected in consortium size and in project budget) on various performance dimensions of publicly funded collaborative research and development (R&D) projects. Recent trends in European research policy are founded on the premise that such scale economies do exist, hence the emphasis placed on the critical mass of R&D projects. We argue that large scale in collaborative R&D confers both positive and negative effects, and thus we hypothesize an inverse U‐shaped relation between scale and performance. We only find an inverse U‐shaped effect of consortium size on networking impacts and a U‐shaped effect of project budget on goal achievement. No other statistically significant relationships were found for the remaining performance dimensions (i.e. scientific outputs, technological outputs, and research capacity impacts). More generally, and consistent with recent work on the relation between quality and quantity in academic research, our analyses offer little evidence to support the idea that increasing scale generally improves collaborative R&D performance. It appears that the simple assumption ‘bigger is better’ in European Union‐financed collaborative R&D may need to be reconsidered.     

A basic model for evaluating R&D performance: theory and application in Italy

Nowadays the public R&D laboratories have a fundamental role in countries' development, supporting businesses as they face the technological challenges in the turbulent world scenarios. Measuring the performance of R&D organisations is crucially important to decisions about the level and direction of public funding for research and development. This research considers the public laboratories like systems and develops a mathematical model based on the measurement of R&D activities with k‐indices. The score obtained from the research laboratories evaluation (relev) methodology synthesises in single value financial, scientific and technological aspects. It is an indicator, for R&D manager and policy maker, of performance in relation to other research organisations or in a time series. The method is an instrument of strategic planning and can be used for the improvement of individual activities and the overall performance of public R&D bodies.     

Integrating the R&D and Patent Functions: Implications for New Product Performance

This paper examines the impact of cross‐functional integration between the research and development (R&D) and the patent functions on new product development (NPD) performance. The attitudinal (collaboration) and the behavioral (contributions of the patent function to NPD) dimension of cross‐functional integration between the R&D and the patent functions are distinguished. It is also investigated if the level of innovativeness moderates the relationship between the attitudinal and the behavioral dimension of cross‐functional integration between the R&D and the patent department and NPD performance. The four hypotheses are tested based on a multi‐informant sample of 101 NPD projects which are nested within 72 technology‐based firms or strategic business units from multiple industries in Germany. The results show that the attitudinal and the behavioral dimensions of cross‐functional integration between the R&D and the patent functions have a significant and positive impact on NPD performance. This lends empirical support for the notion expressed in the literature that certain managerial capabilities are important for understanding the effect of patenting on appropriability outcomes such as value creation and performance. The level of cross‐functional integration between the patent and the R&D functions appears to be one of these critical patent management capabilities that affect the returns from investments into patents. There is support for the hypothesis that the context matters for the effect of cross‐functional integration between the R&D and the patent functions on NPD performance. In line with the initial hypothesis, the level of innovativeness positively moderates the impact of the behavioral dimension of cross‐functional integration between the R&D and the patent department on NPD performance. In contrast to the initial hypothesis, the findings reveal no moderating effect of the level of innovativeness on the link between the attitudinal dimension of cross‐functional integration between the R&D and the patent department and NPD performance. This implies that joint objectives and an open and trustful working relationship between the R&D and the patent functions are not sufficient for achieving higher NPD performance if firms aim to develop very innovative products. In the case of highly innovative products, the actual behavior, that is, the specific contributions of the patent department to the NPD project, matters. Overall, these findings have important implications for improving performance by means of effectively integrating the patent and the R&D functions during NPD.     

A longitudinal study of the individual characteristics of effective R&D project team leaders

One hundred and eighteen project team leaders from five industrial research and development (R&D) organizations were studied to determine the individual characteristics that longitudinally predict leader effectiveness. Hypotheses generated from an interactionist framework and the theory of purposeful work behavior (Barrick et al., 2013 ) found an innovative orientation and job involvement to each predict 1-year later and 5-years later job performance ratings by immediate supervisors. Low need for clarity predicted 1-year later performance ratings. Self-esteem and job involvement each predicted 5-years later profitability of the project, and job involvement predicted project speed to market. As hypothesized, type of R&D work was found to be a moderator whereby an innovative orientation predicted 1-year and 5-years later job performance primarily for research projects, and a low need for clarity predicted 1-year later performance mainly for research projects. Implications for models of interactionism and leader effectiveness in R&D are discussed.     

The value of a ‘failed’ R&D project: an emerging evaluation framework for building innovative capabilities1

In literature and R&D organizations alike, project success consists in minimizing the deviations from set targets in terms of quality, cost and time. The main management task is to execute and monitor progress to reduce risks – assuming that project attributes are known, necessary resources can be estimated and a reasonable time table can be agreed upon. In such a context, evaluating project success is easy. However, in an innovative context, setting project targets initially is difficult and the contributions of the projects sometimes are of an unexpected nature. This paper investigates if projects can be evaluated in terms of how they contribute to the building of innovative capabilities of the firm instead of independently. Based on a case study at the Régie Autonome des Transports Parisians and the theoretical framework of innovation fields, a framework for evaluating projects from an additional perspective is proposed. Based on the following four criteria: financial resources, the development of a structured, refined and expanded strategic vision, developed competences (with related suppliers) and identification of knowledge gaps (occasionally with related partners for knowledge production), this framework shows how seemingly failed R&D projects can instead be considered as invaluable to the overall innovation process.