Unpacking p-Hacking and Publication Bias

• 成果类型: Article
• 署名作者: Brodeur, Abel; Carrell, Scott; Figlio, David; Lusher, Ester
• 署名单位: University of Ottawa; National Bureau of Economic Research; University of Texas System; University of Texas Austin; IZA Institute Labor Economics; National Bureau of Economic Research; University of Rochester; National Bureau of Economic Research; University of Rochester; Pennsylvania Commonwealth System of Higher Education (PCSHE); University of Pittsburgh
• 刊物名称: AMERICAN ECONOMIC REVIEW
• ISSN/ISSBN: 0002-8282
• DOI: 10.1257/aer.20210795
• 发表日期: 2023
• 页码: 2974-3002
• 关键词: transparency

摘要

We use unique data from journal submissions to identify and unpack publication bias and p- hacking. We find initial submissions display significant bunching, suggesting the distribution among published statistics cannot be fully attributed to a publication bias in peer review. Desk- rejected manuscripts display greater heaping than those sent for review; i.e., marginally significant results are more likely to be desk rejected. Reviewer recommendations, in contrast, are positively associated with statistical significance. Overall, the peer review process has little effect on the distribution of test statistics. Lastly, we track rejected papers and present evidence that the prevalence of publication biases is perhaps not as prominent as feared.