How Credible Is the Credibility Revolution?

• 成果类型: Article
• 署名作者: Lang, Kevin
• 署名单位: Boston University; IZA Institute Labor Economics
• 刊物名称: JOURNAL OF LABOR ECONOMICS
• ISSN/ISSBN: 0734-306X
• DOI: 10.1086/732772
• 发表日期: 2025
• 页码: 635-663
• 关键词: false discovery rate; science-wise false; microarray; jager; tests

摘要

Economists analyzing a well-conducted randomized controlled trial or natural experiment and finding a statistically significant effect conclude that the null of no effect is unlikely to be true. But how frequently is this conclusion warranted? The answer depends on the proportion of tested nulls that are true and the test's power. I model the distribution of t-statistics in leading economics journals. Using my preferred model, 65% of narrowly rejected null hypotheses and 41% of all rejected null hypotheses with |t|<10 are likely to be false rejections. For the null to have only a .05 probability of being true requires a t of 5.48.