Measurement of patient-reported outcomes. 1: The search for the Holy Grail

Abstract

Patient-reported outcome measures (PROMs) are used to collect information directly from patients. They may cover several different types of outcomes ranging from symptoms, functioning, utility, satisfaction, through to quality-of-life (QoL). They generally consist of self-completed questionnaires that can be administered by means of hard copies or in a range of electronic formats. PROMs vary considerably in terms of the constructs they assess, the care with which they are developed, and their scientific quality. However, none of the PROMs available approach the quality of measurement achieved by measures/instruments used in physics. PROs are examples of latent variables. These are not directly observable, but can be inferred from, for example, responses to a questionnaire. The only measure of a latent variable that approaches the quality of measurement achieved by the physical sciences is the Lexile Framework for Reading. This framework is based on a construct theory that grew out of an analysis of several available reading measures. A specification equation was generated that was able to link the construct theory to scores obtained with the Lexile measure. A fundamental requirement of this quality of measurement is that the data collected with the model fit Rasch Measurement Theory (RMT). It is argued that PROM developers should aspire to match this level of measurement sophistication if their instruments are to provide valid insights into the impacts of disease and its treatment.     

Measurement of patient-reported outcomes. 2: Are current measures failing us?

Abstract

All instruments designed to measure latent (unobservable) variables, such as patient-reported outcomes (PROs), have three major requirements; a coherent construct theory, a specification equation, and the application of an appropriate response model. The theory guides the selection of content for the questionnaire and the specification equation links the construct theory to scores produced with the instrument. For the specification equation to perform this role, the patient-reported outcome measure (PROM) must employ a response model that generates values for its individual items. The most commonly applied response model in PROM development is the Rasch model. To date this level of measurement sophistication has not been achieved in PRO measurement. Consequently, it is not possible to establish a PROM’s true construct validity. However, the development of the Lexile Framework for Reading has demonstrated that such objective measurement is possible for latent variables. This article argues that higher quality PROM development is needed if meaningful and valid PRO measurement is to be achieved. It describes the current state of PROM development, shows that published reviews of PROMs adopt inappropriate criteria for judging their quality, and illustrates how the use of traditional PROMs can lead to incorrect (and possibly dangerous) conclusions being drawn about the efficacy of interventions.     

Composite outcome measurement in clinical research: the triumph of illusion over reality?

Abstract

Composite measures that combine different types of indicators are widely used in medical research; to evaluate health systems, as outcomes in clinical trials and patient-reported outcome measurement. The potential advantages of such indices are clear. They are used to summarise complex data and to overcome the problem of evaluating new interventions when the most important outcome is rare or likely to occur far in the future. However, many scientists question the value of composite measures, primarily due to inadequate development methodology, lack of transparency or the likelihood of producing misleading results. It is argued that the real problems with composite measurement are related to their failure to take account of measurement theory and the absence of coherent theoretical models that justify the addition of the individual indicators that are combined into the composite index. All outcome measures must be unidimensional if they are to provide meaningful data. They should also have dimensional homogeneity. Ideally, a specification equation should be developed that can predict accurately how organisations or individuals will score on an index, based on their scores on the individual indicators that make up the measure. The article concludes that composite measures should not be used as they fail to apply measurement theory and, consequently, produce invalid and misleading scores.     

An independent assessment of the unidimensionality,reliability, validity and factor structure of the LibQUAL+™ scale

This empirical study provides a thorough measurement analysis of the LibQUAL+? scale for measuring library service quality. In particular, the study assesses the unidimensionality of the scale, the scale's psychometric properties and its factor structure. Data were gathered using a self-administered questionnaire distributed to a sample of university library users: 189 in Canada and 374 in Mexico. Using confirmatory factor analysis (CFA), the study confirms the assumption of unidimensionality in only two of the three current sub-dimensions of the scale. The results of this study also show that the LibQUAL+? scale consists of four dimensions. The remainder of the CFA results in the study provides strong support for the general reliability and validity of the four-dimensional structure.