Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics: Comment
成果类型：
Article
署名作者：
Kranz, Sebastian; PUtz, Peter
署名单位：
Ulm University; University of Bielefeld
刊物名称：
AMERICAN ECONOMIC REVIEW
ISSN/ISSBN：
0002-8282
DOI：
10.1257/aer.20210121
发表日期：
2022
页码：
3124-3136
关键词：
摘要：
Brodeur, Cook, and Heyes (2020) study hypothesis tests from eco-nomic articles and find evidence for p-hacking and publication bias, in particular for instrumental variable and difference-in-difference studies. When adjusting for rounding errors (introducing a novel method), statistical evidence for p-hacking from randomization tests and caliper tests at the 5 percent significance threshold vanishes for difference-in-differnce studies but remains for instrumental variable studies. Results at the 1 percent and 10 percent significance thresh-olds remain largely similar. In addition, Brodeur, Cook, and Heyes derive latent distributions of z-statistics absent publication bias using two different approaches. We establish for each approach a result that challenges its applicability.