Question order bias revisited: A split-ballot experiment on satisfaction with public services among experienced and professional users

• 成果类型: Article
• 署名作者: Thau, Mads; Mikkelsen, Maria Falk; Hjortskov, Morten; Pedersen, Mogens Jin
• 署名单位: Aarhus University; University of Copenhagen
• 刊物名称: PUBLIC ADMINISTRATION
• ISSN/ISSBN: 0033-3298
• DOI: 10.1111/padm.12688
• 发表日期: 2021
• 页码: 189-204
• 关键词: citizen satisfaction; performance; DISCONFIRMATION; expectations; assimilation; CUSTOMER; parents; MODEL

摘要

Public decision-makers increasingly rely on satisfaction surveys to inform budget and policy decisions. Yet, our knowledge of whether, and under what conditions, this input from public service users provides valid performance information remains incomplete. Using a preregistered split-ballot experiment among government grant recipients in Denmark, this article shows that the ordering of survey questions can bias satisfaction measures even for highly experienced and professional respondents. We find that asking about overall satisfaction before any specific service ratings lowers overall user satisfaction, compared to the reverse order, while the correlations between specific ratings and overall satisfaction are relatively stable. Also, the question order effect outweighs that of a large-scale embezzlement scandal, which unexpectedly hit the investigated government agency during the data collection. Our results support rising concerns that subjective performance indicators are susceptible to bias. We discuss how practitioners should approach satisfaction surveys to account for the risk of question order bias.