Beyond choice: investigating the sensitivity and validity of measures of strength of preference

Many experiments investigating different decision theories have relied heavily on pairwise choices between lotteries. These are easy to incentivise, but often yield only limited dichotomous information. This paper considers whether respondents’ judgments about their strength of preference (SoP) for one alternative over another can usefully supplement standard choice data. We report extensive evidence that such judgments show sensitivity to variations in question format and parameter values in the directions we should expect, not only within-subject but also between-sample. We illustrate how such judgments can usefully supplement standard pairwise choice data and enrich our understanding of observed behaviour.