Computational Reproducibility in Finance: Evidence from 1,000 Tests

• 成果类型: Article
• 署名作者: Perignon, Christophe; Akmansoy, Olivier; Hurlin, Christophe; Dreber, Anna; Holzmeister, Felix; Huber, Juergen; Johannesson, Magnus; Kirchler, Michael; Menkveld, Albert J.; Razen, Michael; Weitzel, Utz
• 署名单位: Hautes Etudes Commerciales (HEC) Paris; Centre National de la Recherche Scientifique (CNRS); Universite de Orleans; Stockholm School of Economics; University of Innsbruck; Vrije Universiteit Amsterdam; Tinbergen Institute; Radboud University Nijmegen
• 刊物名称: REVIEW OF FINANCIAL STUDIES
• ISSN/ISSBN: 0893-9454
• DOI: 10.1093/rfs/hhae029
• 发表日期: 2024
• 页码: 3558
• 关键词: cross-section; liquidity risk; stock returns; replication; ECONOMICS; illiquidity; TRANSPARENCY

摘要

We analyze the computational reproducibility of more than 1,000 empirical answers to 6 research questions in finance provided by 168 research teams. Running the researchers' code on the same raw data regenerates exactly the same results only 52% of the time. Reproducibility is higher for researchers with better coding skills and those exerting more effort. It is lower for more technical research questions, more complex code, and results lying in the tails of the distribution. Researchers exhibit overconfidence when assessing the reproducibility of their own research. We provide guidelines for finance researchers and discuss implementable reproducibility policies for academic journals.