RCTs to Scale: Comprehensive Evidence From Two Nudge Units

• 成果类型: Article
• 署名作者: DellaVigna, Stefano; Linos, Elizabeth
• 署名单位: University of California System; University of California Berkeley; National Bureau of Economic Research; University of California System; University of California Berkeley
• 刊物名称: ECONOMETRICA
• ISSN/ISSBN: 0012-9682
• DOI: 10.3982/ECTA18709
• 发表日期: 2022
• 页码: 81-116
• 关键词: transparency; science; BIAS

摘要

Nudge interventions have quickly expanded from academic studies to larger implementation in so-called Nudge Units in governments. This provides an opportunity to compare interventions in research studies, versus at scale. We assemble a unique data set of 126 RCTs covering 23 million individuals, including all trials run by two of the largest Nudge Units in the United States. We compare these trials to a sample of nudge trials in academic journals from two recent meta-analyses. In the Academic Journals papers, the average impact of a nudge is very large-an 8.7 percentage point take-up effect, which is a 33.4% increase over the average control. In the Nudge Units sample, the average impact is still sizable and highly statistically significant, but smaller at 1.4 percentage points, an 8.0% increase. We document three dimensions which can account for the difference between these two estimates: (i) statistical power of the trials; (ii) characteristics of the interventions, such as topic area and behavioral channel; and (iii) selective publication. A meta-analysis model incorporating these dimensions indicates that selective publication in the Academic Journals sample, exacerbated by low statistical power, explains about 70 percent of the difference in effect sizes between the two samples. Different nudge characteristics account for most of the residual difference.