Measuring the Unmeasurable? Systematic Evidence on Scale Transformations in Subjective Survey Data

Ordered response scales are ubiquitous in economics, but their interpretation rests on an untested assumption: that numerical labels reflect equal psychological intervals. The contribution of this paper is to provide a systematic assessment of this linearity assumption, developing a general framework to quantify how easily empirical results can be overturned when it is relaxed. Using original experimental data, we show that respondents use survey scales in ways that deviate from linearity, but only mildly so. Focusing on wellbeing research, we then replicate 40,000+ coefficient estimates across more than 80 papers published in top economics journals. Coefficient signs are remarkably robust to the mild departures from linear scale-use we document experimentally. However, estimates of relative effect sizes, which are crucial for policy applications, are unreliable even under these modest non-linearities.