Rankings Methodology for International Comparisons of Institutions and Individuals: an Application to Economics in Australia and New Zealand

Abstract.  The objectives of this paper are threefold. First, we critically survey and analyse the different methodologies that have been adopted in the rankings literature. Second, using journal articles included in the ECONLIT database and on the basis of two criteria – one based on citations and the other on perceptions of journal quality, we rank economics teaching departments in Australia and New Zealand for 1988–2002 and 1996–2002 and for individual academic economists for the periods 1988–2002, 1988–1995 and 1996–2002. Furthermore, we identify individual star performers and recognize them in a designated 'Hall of Fame' for 1988–2002. Third, our methodology enables us to make international comparisons on total and per capita bases. Previous multi-country ranking studies in economics do not rank economics departments. They provide rankings based on total publications in economics in universities regardless of whether the economists are in the economics departments or in other departments. Thus, no rankings on per capita bases are provided. We correct this shortcoming by providing total and per capita rankings and analysing the correlations between total and per capita rankings. This is the first study to examine whether the size of the economics department matters with regard to productivity.     

Publishing performance of Spanish academics: 1970—2004

This work complements some of the results appearing in the article “Publishing Performance in Economics: Spanish Rankings” by Dolado et al. (Span Econ Rev 5: 80–103, 2003) Specifically we focus on the robustness of the results regardless of the time span considered, the effect of the choice of a particular database on the final results, and the effects on changes in the unit of institutional measure (departments vs. institutions as a whole). Differences are significant when we expand the time period considered. There are also significant but small differences if we combine datasets to derive the rankings. Finally, department rankings offer a more precise picture of the situation of the Spanish academics, although results do not differ substantially from those obtained when overall institutions are considered.     

Media attention and betting markets

This paper investigates whether biased media attention affects perceptions about future events. We use data on World Cup tournaments in alpine skiing for the period of 1992–2014 and exploit close races as a source of randomness for ranking positions. Since the allocation of media attention is skewed towards the winner and athletes on the podium, we can estimate the causal effect of media attention. Our results document that ranking schemes generate sharp discontinuities in media attention even in close competitions. However, both regression discontinuity and instrumental variables estimates reveal that biased media attention neither affects prices nor quantities in the betting market. We conduct a series of robustness tests to explore the sensitivity of our results.     

The becoming of user-generated reviews: Looking at the past to understand the future of managing reputation in the travel sector

The emergence of User-Generated-Content (UGC) is challenging the equilibrium of reputation management practices prompting widespread change and organizational restructuring. Formal accreditation schemes mingle with less formal mechanisms which place users in the middle making them draw their own conclusions about products and services. This paper explores how UGC reviews and ratings have intensified the contingency of organizational reputation in the travel sector. The findings are based upon a corpus of data including: a field study at the offices of the largest travel user-generated website, TripAdvisor, and a netnographic approach. In particular we discuss the shift from Word-Of-Mouth to eWOM, the consequences for the sector and provide a balanced view of the role of reviews, ratings and lists. We are concluding with a conceptual model for managing online reputation in the era of UGC, while acknowledging the current process of transformation in reputation management.     

An updated ranking of the economic research institutions in China (2000–2009)

This paper ranks the economic research institutions in China based on publications in the SSCI economics journals from 2000 through 2009 and finds that the gap in the research capacity of economic research institutions between the mainland and Hong Kong is very large, even though the difference in overall quality is not so significant. This paper can reflect successes and failures of the reform of the economics education in China over the past decade and give important implications for the ongoing reforms as well.     

An exercise in composite indicators construction: Assessing the sustainability of Italian regions

This paper presents a piece of research aimed at evaluating the relative sustainability of the Italian Regions. After selecting a core set of indicators, for which we referred to the EU Sustainable Development Strategy, we built a composite index and checked for its robustness. As a result we got ‘many numbers’, that is, a range of possible rankings for Italian Regions.     

Broadband expectations and the convergence of ranks

Broadband Internet service to widely held to be a significant contributor to economic development and global competitiveness, and comparison of adoption rates across countries are common. This paper presents evidence that the relative broadband Internet adoption ranks across the Organisation for Economic Co-Operation and Development (“OECD”) countries are converging to the wireline telephone adoption ranks in the mid 1990s. This was a time when wireline telephone service had reached maturity, but before consumers began to abandon traditional telephone services for mobile services and Internet telephone technologies. As such, in the absence of better data on household adoption, wireline telephone rank is a useful proxy for a country's ultimate fixed-line broadband penetration rank. Having such an educated guess available regarding broadband rank should reduce the amount of anxiety over rankings, since similar rankings across the two services implies suitable broadband performance. Large departures, alternately, may be a cause for concern or delight. Like prior analyses, the findings suggest that the adoption of communications services is largely an economic and demographic issue.     

Testing inter-group ranking heterogeneity: do patient characteristics matter for prioritization of quality improvements in healthcare service?

In many research contexts, such as social science, marketing, education, psychology and medicine, it is frequently of interest to compare two or more groups of subjects (e.g. people of different gender, age or nationality), who are asked to rank a set of alternatives according to their personal liking or opinion, for investigating the presence of group effect. The common investigation aim is to detect customers with homogeneous preferences (or priorities) in order to serve each group as properly as possible. Several approaches have been proposed in the literature for testing ranking heterogeneity among groups of subjects. This paper focuses on an approach considering diversity as a generalization of the notion of variation and investigates the performance of a testing procedure for ranking heterogeneity based on the index of segregation power. The performance of the testing procedure has been investigated via a Monte Carlo simulation study under several scenarios, differing for group size, number of ranked alternatives and system of hypothesis. Furthermore, using a real data set, the testing procedure is exploited for investigating whether patient age and gender matter for patient prioritization of quality improvement in healthcare service.     

Weighted distances between preferences

Individual disagreements are assumed to be reflected in the preferences. Distance functions, e.g., the well-known Kemeny (1959) metric, are used to measure these disagreements. However, a disagreement on how to rank the top two alternatives may be perceived more (or less) than a disagreement on how to rank the bottom two alternatives. We propose two conditions on functions which characterize a class of weighted semi-metric functions. This class of semi-metrics allows to quantify disagreements according to where they occur in preferences. It turns out one of these functions, “the path minimizing function”, is the only metric which generalizes the Kemeny metric.     

CONSTRUCTING IMPACT FACTORS TO MEASURE THE INFLUENCE OF SUPPLY CHAIN MANAGEMENT AND LOGISTICS JOURNALS

Data from ten supply chain management and logistics journals are combined with data from the ISI Web of Science database to develop a series of impact factors. The resulting constructed impact factors provide a context for comparing and ranking leading academic supply chain management and logistics journals. They also provide an objective way of comparing the relative influence of supply chain management and logistics journals, as well as a platform for comparing their influence with that of journals in more general fields like management and business.