Stability of Experimental Results: Forecasts and Evidence

• 成果类型: Article
• 署名作者: DellaVigna, Stefano; Pope, Devin
• 署名单位: University of California System; University of California Berkeley; National Bureau of Economic Research; University of Chicago
• 刊物名称: AMERICAN ECONOMIC JOURNAL-MICROECONOMICS
• ISSN/ISSBN: 1945-7669
• DOI: 10.1257/mic.20200129
• 发表日期: 2022
• 页码: 889-925
• 关键词: field; replicability; benefits; BIAS

摘要

How robust are experimental results to changes in design? And can researchers anticipate which changes matter most? We consider a real-effort task with multiple behavioral treatments and examine the stability along six dimensions: (i) pure replication, (ii) demographics, (iii) geography and culture, (iv) the task, (v) the output measure, and (vi) the presence of a consent form. We find near-perfect replication of the experimental results and full stability of the results across demographics, significantly higher than a group of experts expected. The results differ instead across task and output change, mostly because the task change adds noise to the findings.