Methods Matter: p-Hacking and Publication Bias in Causal Analysis in Economics

• 成果类型: Article
• 署名作者: Brodeur, Abel; Cook, Nikolai; Heyes, Anthony
• 署名单位: University of Ottawa
• 刊物名称: AMERICAN ECONOMIC REVIEW
• ISSN/ISSBN: 0002-8282
• DOI: 10.1257/aer.20190687
• 发表日期: 2020
• 页码: 3634-3660
• 关键词: FACTS; con

摘要

The credibility revolution in economics has promoted causal identification using randomized control trials (RCT), difference-in-differences (DID), instrumental variables (IV) and regression discontinuity design (RDD). Applying multiple approaches to over 21,000 hypothesis tests published in 25 leading economics journals, we find that the extent of p-hacking and publication bias varies greatly by method. IV (and to a lesser extent DID) are particularly problematic. We find no evidence that (i) papers published in the Top 5 journals are different to others; (ii) the journal revise and resubmit process mitigates the problem; (iii) things are improving through time.