Factors affecting intercoder reliability: a Monte Carlo experiment

Although it has long been a consensus that intercoder reliability is crucial to the validity of a content analysis study, the choice among them has been debated. This study reviewed and empirically tested most popular intercoder reliability indices, aiming to find the most robust index against prevalence and rater bias, by empirically testing their relationships with response surface methodology through a Monte Carlo experiment. It was found that Maxwell’s R.E is superior to Krippendorff’s α, Scott’s π, Cohen’s κ, I _r of Perreault and Leigh, and Gwet’s AC ₁. More nuanced relationships among prevalence, sensitivity, specificity and the intercoder reliability indices were discovered through response surface plots. Both theoretical and practical implications were also discussed in the end.     

Estimating intercoder reliability: a structural equation modeling approach

Intercoder reliability is usually estimated with a summary index, and yet the limitations concerning the indexing approach have been well noted. This study critically reviewed all the existing major modeling approaches to estimating intercoder reliability, and empirically tested and further compared these approaches. It was found that latent variable modeling, also called the second-generation SEM, generally perform better than log-linear modeling, and is able to explain the paradox haunting some indices, and to spot the sources of disagreement among coders. Implications were discussed at last.     

Underlying determinants driving agreement among coders

There are plenty of intercoder reliability indices, whereas the choice of them has been debated. With a Monte Carlo simulation, the determinants of the agreement indices were empirically tested. The chance agreement of Bennett’s S is found to be only affected by the number of categories. Consequently, S is a category based index. The chance agreements of Krippendorff’s $\alpha $ , Scott’s $\pi $ and Cohen’s $\kappa $ are affected by the marginal distribution, the level of difficulty and the interaction between them, and yet the difficulty level influences their chance agreements abnormally. The three indices are hence in general distribution based indices. Gwet’s $AC_1$ reversed the direction of the three aforementioned indices, but its chance agreement is additionally affected by the number of categories and the interaction between the number of categories and the marginal distribution. $AC_1$ can be classified into a class based on the number of categories, the marginal distribution and the level of difficulty. Both theoretical and practical implications were also discussed in the end.     

Mapping leisure shopping trip decision making: validation of the CNET interview protocol

Qualitative research methods can provide an in-depth understanding of how people come to certain decisions, providing valuable input to ground behavioural assumptions in activity-based travel demand models and to implement high impact policy measures to change travel behaviour. The CNET interview protocol is a semi-structured personal interview method to elicit the mental representation of individuals’ decision making. There is a risk of bias caused by the interviewer’s interpretation of the respondents’ answers. Therefore, the quality of the CNET interview protocol is assessed by evaluating its trustworthiness using intercoder reliability tests. Krippendorff’s alpha is identified as the most appropriate measure. The intercoder reliability is sufficiently high. Consequently, the CNET interview protocol can be considered a valid method to measure and map individuals’ considerations in complex spatio-temporal decision problems.     

Cross Validation Calibration Based Shrinkage – Revisited for the Construction of Prognostic Indices

The use of shrinkage methods for the construction of prognostic indices has been paid increasing attention in the literature on medical statistics in the last years. One approach for the construction of a shrinkage factor is cross validation calibration as suggested by van H ouwelingen and le C essie (1990). We investigate this approach in more detail. First we try to clarify why shrinkage factors constructed by cross validation calibration tend to be smaller than 1. Second we explain why use of this shrinkage factor can result in an improvement of the average prediction error. Third we investigate the possible gain for constellations relevant in medical research by means of a simulation study, focusing on the dilemma, that the improvement on average has to be paid by distinct deteriorations for some patients. Finally we conclude that it is necessary to rethink the choice of loss functions in constructing prognostic indices before recommendations about the use of shrinkage methods can be made.     

基于模糊状态的串-并联可修系统的模糊可靠性

研究了具有3台修理设备且在假定关键部件具有优先修理权的可修串-并联系统。利用马尔可夫过程对系统的各状态进行分析,得出系统各状态的概率。然后利用基于模糊状态的可靠性理论,通过定义系统模糊工作以及模糊故障的隶属函数,得到了可修串-并联系统的模糊可靠性指标的计算公式,并比较其结果与传统可靠性理论的不同之处。     

Cointegration of price measures: Evidence from the G-7

This study addresses index-dependency of empirical results associated with the purchasing power parity (PPP) relationship. Using four key price indices involving the G-7 nations, empirical tests for long-run co-movement are conducted. A test for linear restrictions is imposed. The speeds of adjustment are calculated for statistically significant linear combinations. The speed of the short-run response to disequilibrium differs both within and across countries. The seven-country average reveals that the CPI has the quickest recovery response to a one-time disturbance. The findings suggest that PPP results are not dependent upon the choice of index when an explicit set of indices is cointegrated.(JEL F3)     

Association,agreement, and equity

The paper attempts to make a clear distinction between three broad families of statistical indices: association, agreement, and what one may call equity. The need for this distinction arises in social research, for example, where reliability (accuracy, reproducibility, and stability) is assessed by measures of association rather than agreement. In this application, the assumptions built into an association measure conflict with the reality that gives rise to reliability data. A second motivation for this distinction is that association measures tend to express chance as the product of two potentially very different frequency distributions, agreement as the product of two identical distributions, and equity ignores such distributions altogether. A third motivation for this distinction is that the probability distribution of such measures does not depend on whether they are linear or non-linear, symmetrical or asymmetrical, or whether they express predictability or the extremality of a frequency distribution, but on their family membership. Notions of association, agreement, and equity have inherently nothing to do with the (nominal, ordinal, interval, and ratio) ordering in data. The 2-by-2 case is therefore chosen as the basis of the proposed distinction. All statistical indices, whether they are designed to characterise multivariate data or to identify complex orderings, ought to be applicable to this most reduced case of two variables, making one distinction in each. To test a coefficient's membership in one of the three families, nothing more complex is needed.     

Stablecoins as diversifiers,hedges and safe havens: A quantile coherency approach

This study investigates the diversifier, hedge and safe haven properties of stablecoins against various financial assets including cryptocurrencies such as Bitcoin, Ether, XRP and stock market indices. Using quantile coherency we show that stablecoins included in the study act as weak hedges in normal conditions and weak safe havens when considering moments of market turmoil and there is little evidence to support the existence of any contagion effects between the cryptocurrency and stablecoin markets. Aforementioned results are not significantly influenced by the choice of investment horizon. We further evaluate the implications of those results for the question of whether stablecoins are in fact stable.     

会计政策选择研究述评

会计政策选择是财务会计研究的重要内容。目前会计政策选择研究主要集中在上市公司的会计政策选择，采用规范研究、实证研究的方法对会计政策选择的原因、影响因素、动机、经济后果等进行研究。本文主要对国内外会计政策选择研究状况进行述评，以期为会计政策选择研究提供参考。     

Does designing environmental sustainability disclosure quality measures make a difference?

Assessing the quality of information disclosed by companies is a complex task. Accounting studies usually rely on analysing the content of corporate reports using measures to obtain a proxy for the information reported by companies. However, there is no consensus about the best design for these measures. The objective of the current paper is to investigate if there are significant differences in the results generated from seven alternative measures for assessing the quality of FTSE100 environmental sustainability reporting. Seven measures/indices have been used to assess disclosure quality. The three unidimensional measures include two “quantity measures” and one “scope measure” that measure the volume and width/coverage of information, respectively. Three compound measures are adopted from the literature, and the final measure is a multidimensional quality model, based on the results of a questionnaire ascertaining the perceptions of 86 preparers and 177 users of annual report (AR) and/stand‐alone corporate responsibility report (CRR). Although the results of the empirical analysis indicate that the measures are significantly correlated, the choice of a specific method can result in the very different ranking of companies. The evidence presented indicates that the choice of measure is of key importance.     

Basic Process Capability Indices: An Expository Review

A review of the four basic process capability indices has been made. The interrelationship among these indices has been highlighted. Attention has been drawn to their drawbacks. The relation of these indices to the proportion nonconforming has been dwelt upon and the requirement of the adequate sample size has been emphasized. Cautionary remarks on the use of these indices in the case of nonnormal distributions, skewed distributions, and autocorrelated data are also presented. The effect of measurement error on process capability indices has been dealt with in great detail.     

基于可靠性优化的供应链软联盟规模研究

建立了供应链整体可靠度与其软联盟规模及功能企业选择方式之间的数学模型,并给出了以供应链整体可靠度最大为目标时,供应链软联盟规模一定时,功能企业的选择方式;或在选择方式一定时,供应链软联盟规模的确定方法。     

Retaining the visitor,enhancing the experience: identifying attributes of choice in repeat museum visitation

• How people make choices in relation to cultural and leisure consumption has been explored from the viewpoint of motivation, lifestyle segmentation, and lifecycle. Little is known about the specific characteristics associated with choices to visit, re‐visit, or not to visit a museum. Understanding characteristics of choice, developing incentives, bundled packages, and levels of pricing is an essential element in marketing strategies for museums operating in a competitive leisure marketplace. However, determining what really matters to cultural consumers is complex and methodologies to assist in unraveling such complexities are not easily identified. This study aimed to address ways in which people respond to specific incentives as influences in choosing museum visitation. The study was conducted in two major museums in Australia to determine how useful choice modeling is in identifying features that matter to cultural consumers. The results suggest that choice modeling has much to offer in relation to understanding the benefits people are seeking from a museum experience as well as offering strategic insight into potential collaborative ventures and re‐combinations of existing museum products and services.

Copyright © 2008 John Wiley & Sons, Ltd.     

On the fundamental thought behind voting rules

The theme is Arrow's requirement in his theorem of 1951 on methods for group choice, that the choice be independent of irrelevant alternatives. The attention is drawn to (1) his own explanation of this requirement in 1972, which is a quite different understanding than has been discussed in the voluminous literature on the theorem, (2) that Arrow, in fact, in 1985 showed an understanding for how irrelevant alternatives might in a meaningful way influence the group choice, (3) that admittedly the border-line between irrelevant and relevant alternatives in Arrow's original statement is arbitrary, and (4) that Arrow, if he had observed the final thought in the origin of the group theory by Borda, which he admittedly did not, might have realized that Borda's method stringently estimates the relevance of each alternative for the result.The author expresses his surprise that a theoretical conclusion based on an arbitrary fundament has been admired so long.     

Re-innovation: The construct, measurement, and validation

There has been substantial research in the area of innovation, which is widely regarded as a powerful weapon to create competitive advantage. One important reason of the substantial research in innovation is the presence of valid and reliable measures of key innovation characteristics, such as radical, incremental, or disruptiveness innovation. Re-innovation, an extension of innovation, is renowned for its potential in creating competitive advantage with reduced cost and time implications. Unfortunately, very little research in re-innovation has been conducted, possibly because to date it still lacks a psychometrically valid measure. There is also a lack of research on convergent and discriminant validities of the re-innovation construct, resulting in the impossibility to discriminate re-innovation characteristics from radical or incremental innovation characteristics. The purpose of this paper is to develop a valid and reliable scale for measuring the construct of re-innovation. To test its reliability and discriminant and convergent validities, we used data collected from 294 general managers of SBUs in 89 electronics-related companies in Taiwan and performed a series of analyses. The reliability measures, exploratory factor analyses, confirmatory factor analyses, and convergent and discriminant validity tests strongly support our scale. In addition, nomological validity is demonstrated. Hence, this study distinguishes the re-innovation from radical and incremental innovation constructs. Finally, discussion, limitations, and future research are presented.     

The reliability of constructs derived from attitude-behavior theories: an application of generalizability theory in the health sector

Theories of reasoned action, interpersonal behavior and self-efficacy have been frequently used for research into health-related behavior. The present study investigated two methods for assessing the reliability of their theories' constructs: the test-retest method and generalizability theory. Firstly, it is demonstrated with a numerical example that the traditional test-retest coefficient has an anatomical link with the generalizability coefficient. Secondly, generalizability theory was applied in order to identify the number of occasions each construct must be measured to assure its reliability. This procedure was applied to the data collected in a study investigating the psychosocial factors influencing exercising and smoking behaviors among pregnant women. Measuring the constructs of direct attitude, habit, and intention on only one occasion was sufficient to ensure high levels of reliability. Moreover, reliability of the other constructs would be ensured if they were measured on more than one occasion.     

Inconsistencies Across Three Contextual Meanings of Reliability

The purpose of this article is to assess the nature of reliability and its inconsistent definitions across three contextual (conceptual, measurement and statistical) levels under the traditional true score theory. Due to such inconsistencies, two existing quantitative approaches (using r and covariance) are not uniformly understood in Psychology and other disciplines; consequently, their applications to measurements and testings are limited to ambiguous interpretations at the conceptual and measurement levels. To examine the extent of this problem, a questionnaire including various contextual definitions and interpretations of reliability in the literature was distributed in a nationwide survey. Results from six groups of experts representing editors, professors and advanced graduate students in both quantitative and clinical areas indicate that all subject groups generally agreed that a reliable instrument possesses the characteristics of the repeatability of responses of all test-takers at the conceptual level, and the reproducibility of the instrument with little or no variations from the underlying true scores at the measurement level. However, between the editors and noneditors, the endorsements of the common definition at the measurement level show obvious discrepancies. Further, at the statistical level, significant differences were found not only between but also within subject-groups in their interpretations of product-moment correlations and Alpha coefficients for the assessment of reliability at the conceptual and measurement levels. The causes of such inconsistencies were discussed in terms of the inherent limitations of the two statistical approaches used and their insufficiencies for indexing the conceptual and measurement meanings of reliability. Finally, this paper called for developing new statistical indices that are coherent with conceptual and measurement definitions. Before such development, the capacities of existing reliability indices shall be redefined and their application qualifications shall be proportionally re-established for educational, research and clinical purposes.     

浅析汽车产品可靠性工程及其构建

汽车可靠性工程学是把可靠性工程学的最基本的方法和原理结合汽车工程的问题,形成以研究汽车可靠性设计、分配、预测、试验、评价、维修性等为主要内容的应用型科学。本文阐述了汽车产品可靠性工程内容及设计构建的方案。     

An instrument for collecting informed opinions

For collecting informed public opinions a special kind of questionnaire has been developed: the Choice Questionnaire. With this questionnaire, information relating to a decision problem is made available to respondents. This information includes the various options from which a choice has to be made, and the consequences of these options. The Choice Questionnaire also contains a procedure that is designed to assist participants in processing this information and in making a choice based on it. This paper studies the extent to which the Choice Questionnaire is a useful instrument for collecting informed opinions and the effects the instrument has on the decision-making process of the participants. The factors influencing its performance are also considered. This evaluation study has been carried out in a real life context: the choice made by the Dutch population concerning the further application of nuclear power.