Contingency Factors in the Effects of Rater Training on Interrater Agreement: Some Lint in the Bellybutton of Past Research

Researchers, examining the need to train performance appraisal raters, have typically used a complex rating task with several dimensions. However, results of a recent training survey (Mealiea and Duffy, 1986) showed that a number of Canadian organizations use a simple one-item, global rating of employee performance, and they are satisfied with their rating process. The present study looks at the influence of the rating task and the type of measurement on the relationship between rater training and rater effectiveness. Subjects were 105 (approximately 50 per cent female and 50 per cent male) patrons of a tavern in upstate New York who served as contestants in a contest of belly button beauty. Four judges, randomly selected from the audience, served over a three week period. Interrater reliability was assessed using a version of the intraclass correlation coefficient, and the Spearman Brown formula was used to estimate the mean reliability of the four judges. The results indicated an absence of leniency and central tendency bias and high interrater reliability without benefit of training. During the three weeks of the study, the average interrater reliabilities were .85, .99, and .99, respectively. The results, as hypothesized, extend the rater training and measurement literature and can be most parsimoniously explained by adding a contingency factor of task/scale characteristics to the rater training effectiveness theory. Résumé Les chercheurs qui ont examiné le besoin de formation pour les évaluateurs de performance ont toujurs utilisé une tǎche d'évaluation complexe au cours de laquelle plusieurs dimensions étaient évaluées. Cependfant les résultats d'un sondage récent (Mealiea et Duffy, 1986) ont démontré qu'un grand nombre d'entreprises canadiennes utilisent un système global à composante unique pour l'évaluation de la performance et sont satisfaits de cette méthode. La présente étude évalue l'influence de la tǎche d'évaluation et le type de mesure sur la relation entre la formation de l'évaluation et son efficacité. Les sujets de cette étude étaient formés de 105 clients (moitié hommes et moitié femmes approximativement) d'un bar de l'état de New York qui participaient à un concours de beauté ombilicale. Quatre juges choisis au hazard dans la salle ont servi pendant une période de trois semaines. La fiabilité a été évaluée à l'aide d'une version du coefficient de correlation interclasse et la formule Spearman Brown a été utilisée pour estimer la fiabilité moyenne des quatre juges. Les résultats ont démontré l'absence de laxisme et de préjugé à tendance centrale et une fiabilité relative élevées des évaluateurs. Pendant les trois semaines de l'étude, les moyennes de fiabilité étaient de .85, .99, et .99 respectivement. Les résultats, tel que prévu confirment les ouvrages concernant la formation et l'évaluation des évaluateurs et peut ětre expliquée avec grand parcimonie en ajoutant un facteur de contingentement de caractéristiques tǎche/ échelle a la théorie d'efficacité de formation de l'évaluateur.     

Local Exchange Trading Systems: A Solution to the Employment Dilemma?

Local exchange trading systems (LETS) are fast becoming a significant socio-economic phenomenon. It is held that they represent new forms of work which reduce unemployment. The author discusses LETS in the context of three contributions to the future of work to show that LETS are a neglected phenomenon. He then considers whether these proposals on the future of work offer an alternative to the unemployed. The empirical evidence shows a strong bias towards those people in gainful employment and those who are well-educated and well-off. In conclusion the author suggests ways in which this bias may be remedied and measures through which LETS may make a practical contribution to the future of work.     

Interim Realities

Abstract

This paper presents findings from a study of information technology implementation practices in National Health Service hospitals in England. The results suggest that there is a general direction of travel, which involves the progressive linking together of individual systems, so that they are interoperable. We argue that the findings are consistent with meta-governance arguments, but that it is necessary to complement this perspective with an understanding of the nature of information technologies in order to understand them properly. We suggest that ‘interim systems’ will be the reality on the ground for many health care organizations for the foreseeable future.     

Institutional Normativity and the Evolution of Morals: A Behavioural Approach to Ethics

This article explores the normative nature of institutions. The starting point of my investigation is Kahneman, Knetsch and Thaler’s notion of the reference transaction from which I derive a recursive relationship between normative judgements and social practices (i.e. regular, routinised actions in a social group), an implication of which I call the “self-justification of practices”. Drawing on John Dewey, I demonstrate how prevailing practices influence normative standards and thus how institutions become normative entities. I then show how, despite the conservative bias of normative standards, institutional change comes about. Finally, I enquire into the possibility of normative critique of prevailing practices and institutions.     

Transforming economics into what? Heterodox economics and critical realism

The volume under review consists of comments on critical realismby heterodox economists. This paper addresses a number of themesfrom the book, e.g., history and reflexivity, sometimes expandingon the authors' comments on critical realism, other times respondingto them on behalf of critical realists. It considers the goalsof critical realism and ask to what extent the book furthersthem. It concludes that the high level of abstraction at whichcritical realism operates encourages those commenting on itto do so at a similarly high level of abstraction. As it stands,critical realism and the contributions to the book are too divorcedfrom the concerns of concrete economic theorising to be of greatuse to practising heterodox economists.