CHANNELING FISHER: RANDOMIZATION TESTS AND THE STATISTICAL INSIGNIFICANCE OF SEEMINGLY SIGNIFICANT EXPERIMENTAL RESULTS

• 成果类型: Article
• 署名作者: Young, Alwyn
• 署名单位: University of London; London School Economics & Political Science
• 刊物名称: QUARTERLY JOURNAL OF ECONOMICS
• ISSN/ISSBN: 0033-5533
• DOI: 10.1093/qje/qjy029
• 发表日期: 2019
• 页码: 557-598
• 关键词: perry preschool; multiple

摘要

I follow R. A. Fisher's The Design of Experiments (1935), using randomization statistical inference to test the null hypothesis of no treatment effects in a comprehensive sample of 53 experimental papers drawn from the journals of the American Economic Association. In the average paper, randomization tests of the significance of individual treatment effects find 13% to 22% fewer significant results than are found using authors' methods. In joint tests of multiple treatment effects appearing together in tables, randomization tests yield 33% to 49% fewer statistically significant results than conventional tests. Bootstrap and jackknife methods support and confirm the randomization results.