Research performance evaluation of Chinese university: A non-homogeneous network DEA approach

Performance evaluation for universities or research institutions has become a hot topic in recent years. However, the previous works rarely investigate the multiple departments’ performance of a university, and especially, none of them consider the non-homogeneity among the universities’ departments. In this paper, we develop data envelopment analysis (DEA) models to evaluate the performance of general non-homogeneous decision making units (DMUs) with two-stage network structures and then apply them to a university in China. Specifically, the first stage is faculty research process, and the second stage is student research process. We first spit each DMU (i.e. department) into a combination of several mutually exclusive maximal input subgroups and output subgroups in terms of their homogeneity in both stages. Then an additive DEA model is proposed to evaluate the performance of the overall efficiency of the non-homogeneous DMUs with two-stage network structure. By analyzing the empirical results, some implications are provided to support the university to promote the research performance of each department as well as the whole university.     

Assessing the technological innovation efficiency of China's high-tech industries with a two-stage network DEA approach

As significant strategic players in China's economy, high-tech industries need to evaluate and analyze the technological innovation activities from a system point of view to understand and improve their technological innovation efficiency and, thereby, promote their development. Different high-tech industries have different characteristics and thus benefit from different industrial development policies. However, few studies to date have discussed this issue from a systematic perspective. In this study, technological innovation activities are divided into a research and development (R&D) stage and a commercialization stage. A high-tech industrial evaluation framework of technological innovation efficiency based on two-stage network data envelopment analysis (DEA) is constructed with shared inputs, additional intermediate inputs, and free intermediate outputs. Our empirical results indicate that the overall efficiency of most industries is relatively low and the differences between the five high-tech industries (i.e., sub-sectors) we examined are large. The Spearman correlation shows that overall efficiency and R&D efficiency are more correlated than overall efficiency and commercialization efficiency. Additionally, R&D has better average efficiency. The sub-sector with the highest average efficiency is computers and office equipment, and the one with the lowest average efficiency is medicines. These findings indicate the inadequacy but potential for breakthroughs in the evolution of high-tech industries in China. The analysis proves that it is necessary to create different industrial policies to encourage effective progress in certain high-tech industries, and some guidelines for doing so are provided.     

R&D performance assessment of industrial enterprises in China: A two-stage DEA approach

With the growing expenditure on the R&D activities of industrial enterprises above designated size (IEDSs) in China, it is important to evaluate the R&D efficiency of the Chinese IEDSs. However, few studies about R&D efficiency measurement of Chinese IEDSs have considered the internal structure of the R&D production process. To fill this gap, this paper investigates the R&D performance of IEDSs of 30 sample provinces on China's mainland from 2009 to 2014, based on a two-stage data envelopment analysis (DEA) model. The major findings from the empirical results are shown as follows: (i) serious imbalance exists in the R&D resources among Chinese IEDSs of 30 provinces; (ii) there is a decline in the average overall efficiencies after 2012; (iii) there are great differences regarding the performances of R&D activities among the Chinese IEDSs of 30 provinces; (iv) high attention to the R&D activities or strong scientific research atmosphere may promote the R&D efficiency of Chinese IEDSs; and (v) the IEDSs with the relatively high profitability or high government support in terms of R&D activities have relatively poor performance. Based on these findings, several policy suggestions are proposed for the R&D activities of Chinese IEDSs.     

A cautionary note on methods of comparing programmatic efficiency between two or more groups of DMUs in data envelopment analysis

In some applications of data envelopment analysis (DEA) there may be doubt as to whether all the DMUs form a single group with a common efficiency distribution. The Mann–Whitney rank statistic has been used to evaluate if two groups of DMUs come from a common efficiency distribution under the assumption of them sharing a common frontier and to test if the two groups have a common frontier. These procedures have subsequently been extended using the Kruskal–Wallis rank statistic to consider more than two groups. This technical note identifies problems with the second of these applications of both the Mann–Whitney and Kruskal–Wallis rank statistics. It also considers possible alternative methods of testing if groups have a common frontier, and the difficulties of disaggregating managerial and programmatic efficiency within a non-parametric framework.      

The benefits of non-radial vs. radial super-efficiency DEA: an application to burden-sharing amongst NATO member nations

Using radial super-efficiency data envelopment analysis (DEA) has improved the discriminating performance across efficient decision-making units (DMUs). This paper extends the super-efficiency approach to a non-radial super-efficiency DEA (NRSE-DEA) index. NRSE-DEA is shown to be invariant to units of input (output) measurement. NRSE-DEA is illustrated here via an application to NATO burden-sharing assessment in which the DMUs are the member nations of NATO. The NRSE-DEA provides additional insights into the ranking of efficient countries, suggesting which are absorbing a particularly large share of NATO responsibilities. The NRSE-DEA generates a smaller set of efficient DMUs. This, in turn, provides more discriminatory power, a more accurate measure of super-efficiency, a more meaningful ranking of the efficient burden sharing countries, and a more reliable assessment of contributions by NATO members, amongst other policy issues.     

Research efficiency analysis of China's university faculty members: A modified meta-frontier DEA approach

Traditionally, data envelopment analysis (DEA) requires all decision-making units (DMUs) to have similar characteristics and experiences within the same external conditions. In many cases, this assumption fails to hold, and thus, difficulties will be encountered to some extent when measuring efficiency with a standard DEA model. Ideally, the performance of DMUs with different characteristics could be examined using the DEA meta-frontier framework. However, some of these DMUs are mixed-type DMUs that may affiliate with more than one group. Furthermore, the total number of observations of these mixed-type DMUs is limited. This is one of the common problems when studies focus on faculty research performance in higher education institutions. In general, a faculty member is affiliated with a certain department, and if the departmental assessment policy is not suitable for faculty members who are involved in interdisciplinary research, their performance could be underestimated. Therefore, the proposed model is an extension of the DEA meta-frontier framework that can assess the performance of mixed-type DMUs by constructing the reference set without the same type of DMUs. In this paper, the scientific research efficiency of faculty members at the Inner Mongolia University is used as an example to provide a better understanding of the proposed model. The proposed model is intended to provide a fair and balanced performance assessment method that reflects actual performance, especially for mixed-type DMUs.     

DEA efficiency in agriculture: Measurement unit issues

The selection of inputs and outputs in DEA models represents a vibrant methodological topic. At the same time; however, the problem of the impact of different measurement units of selected inputs is understated in empirical literature. Using the example of Czech farms, we show that the DEA method does not provide consistent score estimates, neither a stable ranking for different popular measurements of labour and capital factors of production. For this reason, studies based on DEA efficiency results for differently measured inputs should be compared only with great caution.     

The impact of the economic crisis on the (in)efficiency of public Higher Education institutions in Southern Europe: The case of Spanish universities

The evaluation of university efficiency in Europe began timidly when the European Higher Education Area (EHEA) was created. However, this issue is currently becoming increasingly important in Southern European countries, where the limitation of public funding following the economic crisis in 2008 has put greater pressure on their public universities to achieve excellence and improve competitiveness. In this context, the goals of this paper are: first, to measure the relative technical (in)efficiency of Spanish public Higher Education Institutions in the period 2002–03 to 2012–13, comparing the situation before and during the last economic crisis; and, second, to analyze the determinants of university (in)efficiency and, especially, the direct impact of the crisis. After applying the two-stage double bootstrap DEA methodology, the results show that Spanish public universities have become more efficient during the crisis than before it. In fact, the regression analysis confirms that the “crisis” variable has had a statistically significant positive impact on university efficiency. We also find that age has favorably influenced how these institutions utilize their resources to produce teaching and research outputs, but technical specialization has had a negative effect. Moreover, the regional location of public universities has been also a crucial determinant of their efficiency level. Our findings are therefore relevant for political and academic decision-makers in order to know if public universities have been adequately managed in the crisis period and to identify factors that could improve their efficiency, and hence to help them to enhance their international competitiveness in the future.     

Measuring productivity of research in economics: A cross-country study using DEA

We measure productivity in leading edge economic research by using data envelopment analysis (DEA) for a sample of 21 countries belonging to the Organization for Economic Cooperation and Development (OECD). Publications in ten top journals of economics from 1980 to 1998 are taken as the research output. Inputs are measured by R&D expenditure, the number of universities with economics departments and (as an uncontrollable variable) population. Under constant returns-to-scale, the US emerges as the only efficient country. Under variable returns-to-scale, the efficiency frontier is defined by the US, Ireland and New Zealand. With the exception of the US, all countries in our sample display increasing returns-to-scale, and thus have the potential to raise their efficiency by scaling up their research activities.     

Dynamically evaluating the comprehensive efficiency of technological innovation and low-carbon economy in China's industrial sectors

Technological innovation and low-carbon economy are significant for the high-quality development of China's industrial sectors. However, few scholars combine the two stages closely and discuss their coordinated development. This paper establishes an evaluation index system of technological innovation and low-carbon economy in China's industrial sectors. The technological innovation efficiency, low-carbon economy efficiency, and comprehensive efficiency of technological innovation and low-carbon economy are dynamically investigated by the two-stage data envelopment analysis (DEA) model and the DEA window analysis with 35 subsectors panel data during 1996–2018. The inter-industrial differences in the technological innovation efficiency and low-carbon economy efficiency are considered, and the influencing factors of the comprehensive efficiency of technological innovation and low-carbon economy are studied by the bootstrap truncation regression. The results show that: (1) The development of the technological innovation and low-carbon economy is uncoordinated, and the low-carbon economy efficiency needs improvement; (2) There is heterogeneous of the technological innovation efficiency and low-carbon economy efficiency in the 35 subsectors; (3) The density of science and technology institutions, and the average enterprises scale are positive to the comprehensive efficiency of technological innovation and low-carbon economy, while excessive reliance on technology introduction has a negative impaction. The corresponding suggestions are provided for promoting technological innovation efficiency and low-carbon economy efficiency of industrial sectors.     

A combination of QFD and imprecise DEA with enhanced Russell graph measure: A case study in healthcare

Quality function deployment (QFD) is a proven tool for process and product development, which translates the voice of customer (VoC) into engineering characteristics (EC), and prioritizes the ECs, in terms of customer's requirements. Traditionally, QFD rates the design requirements (DRs) with respect to customer needs, and aggregates the ratings to get relative importance scores of DRs. An increasing number of studies stress on the need to incorporate additional factors, such as cost and environmental impact, while calculating the relative importance of DRs. However, there is a paucity of methodologies for deriving the relative importance of DRs when several additional factors are considered. Ramanathan and Yunfeng [43] proved that the relative importance values computed by data envelopment analysis (DEA) coincide with traditional QFD calculations when only the ratings of DRs with respect to customer needs are considered, and only one additional factor, namely cost, is considered. Also, Kamvysi et al. [27] discussed the combination of QFD with analytic hierarchy process–analytic network process (AHP–ANP) and DEAHP–DEANP methodologies to prioritize selection criteria in a service context. The objective of this paper is to propose a QFD–imprecise enhanced Russell graph measure (QFD–IERGM) for incorporating the criteria such as cost of services and implementation easiness in QFD. Proposed model is applied in an Iranian hospital.     

Are all Scales Optimal in DEA? Theory and Empirical Evidence

Policy recommendations concerning optimal scale of production units may have serious implications for the restructuring of a sector. The piecewise linear frontier production function framework (DEA) is becoming the most popular one for assessing not only technical efficiency of operations, but also for scale efficiency and calculation of optimal scale sizes. The main purpose of the present study is to investigate if neoclassical production theory gives any guidance as to the nature of scale properties in the DEA model, and empirically explore such properties. Theoretical results indicate that the DEA model may have more irregular properties than usually assumed in neoclassical production theory, concerning shape of optimal scale curves and the M-locus. The empirical results indicate that optimal scale may be found over almost the entire size variations in outputs and inputs, thus making policy recommendations about efficient scale difficult. It seems necessary to establish the nature of optimal scale before any practical conclusions can be drawn. Proposals for indexes characterizing the nature of optimal scale are provided.     

The geographical efficiency of education and research: The ranking of U.S. universities

This paper proposes to study the American efficiency of educational diffusion and research productivity following two distinctions: urban vs. rural areas and public vs. private universities. Following this geographical consideration, knowledge diffusion seems to be homogeneous over the American territory, whereas research productivity is more heterogeneous: American research efficiency decreased of 7% points, due to some rural university localizations. Universities in urban areas favor educational quality through high student selection criteria, contrary to those located in more rural areas. Third, public universities present higher educational efficiency, in favoring educational quality over research productivity: the lesser research efficiency of public institutions comes from difficulties in the management of several campuses, by comparison with the private institutions which are all single-campus.     

The efficiency of schools in developing countries,analysed through PISA 2012 data

This paper analyses the efficiency of more than 6800 schools in 28 developing countries, by means of a two-stage Data Envelopment Analysis and using data from OECD PISA 2012. The results reveal that the average efficiency of schools is around 70%, meaning that achievement scores can be raised by 30% by employing a more effective use of available resources. Substantial heterogeneity exists both between and within countries; in general, efficiency scores are higher when comparing schools only with those in the same country. Among factors associated with schools' efficiency, the most important are the characteristics of student population (for instance, motivation, truancy, etc.). Therefore, some practices related to accountability, teachers’ involvement and professional development, and extracurricular activities are also positively associated with higher levels of efficiency.     

Application of evidential reasoning approach and OWA operator weights in road safety evaluation considering the best and worst practice frontiers

Data envelopment analysis (DEA) has recently become relatively popular with road safety experts. Therefore, various decision-making units (DMUs), such as EU countries, have been assessed in terms of road safety performance (RSP). However, the DEA has been criticized because it evaluates DMUs based only on the concept of self-assessment, and, therefore does not provide a unique ranking for DMUs. Therefore, cross efficiency method (CEM) was developed to overcome this shortcoming. Peer-evaluations in addition to self-evaluation have made the CEM to be recognized as an effective method for ranking DMUs. The traditional CEM is based only on the standard CCR (Charnes, Cooper and Rhodes) model, and it evaluates DMUs according to their position relative to the best practice frontier while neglecting the worst practice frontier. However, the DMUs can also be assessed based on their position relative to the worst practice frontier. In this regard, the present study aims to provide a double-frontier CEM for assessing RSP by taking into account the best and worst frontiers simultaneously. For this purpose, the cross efficiency and cross anti-efficiency matrices are generated.Even though a weighted average method (WAM) is most frequently used for cross efficiency aggregation, the decision maker's (DM) preference structure may not be reflected. For this reason, the present study mainly focuses on the evidential reasoning approach (ERA), as a nonlinear aggregation method, rather than the linear WAM. Equal weights are often used for cross efficiency aggregation; consequently, the effect of the DM's subjective judgments in obtaining the overall efficiency is ignored. In this respect, the minimax entropy approach (MEA) and the maximum disparity approach (MMDA) are applied for determining the ordered weighted averaging (OWA) operator weights for cross efficiency aggregation. The weighted cross efficiencies and cross anti-efficiencies are then aggregated using the ERA. Finally, the proposed method, called DF-CEM-ERA, is used to evaluate the RSP of EU countries as well as Serbian police departments (PDs).