Do Not Put All of Your Eggs in One Basket: Multiverse Analysis in Applied Psychology

• 成果类型: Article; Early Access
• 署名作者: Mazei, Jens; Rudolph, Cort W.; Zacher, Hannes; Hueffmeier, Joachim
• 署名单位: Dortmund University of Technology; Wayne State University; Leipzig University
• 刊物名称: JOURNAL OF APPLIED PSYCHOLOGY
• ISSN/ISSBN: 0021-9010
• DOI: 10.1037/apl0001291
• 发表日期: 2025
• 关键词: multiverse analysis; specification curve analysis; open science; TRANSPARENCY

摘要

A multiverse analysis allows researchers to systematically evaluate the support for a hypothesis across a range of sensible ways in which data can be prepared for statistical analysis and/or be analyzed. Accordingly, multiverse analysis provides insights into the relevance of different approaches to, for instance, dealing with outliers or attrition, creating scales, or using different measures for the same construct. The goal of this article is to illustrate the usefulness of multiverse analysis for research in applied psychology and to guide researchers in conducting a multiverse analysis. To do so, we provide a detailed process model of the typical stages involved in conducting a multiverse analysis (along with a shortened version depicting multiverse analysis at a glance), as well as a designated, corresponding preregistration template for multiverse analysis. To showcase the merits of a multiverse analysis, we also evaluate two exemplary hypotheses regarding employees' experience of commuting to and from work. We observed that the results of these hypothesis tests varied strongly depending on how common decisions were made. As such, multiverse analysis represents an important tool for exploring the robustness of knowledge at the level of individual studies, even before a replication is conducted. Hence, multiverse analysis can strengthen the transparency and openness of empirical work.