Detecting and Differentiating the Direction of Change and Intervention Effects in Randomized Trials

• 成果类型: Article
• 署名作者: Bodner, Todd E.; Bliese, Paul D.
• 署名单位: Portland State University; University of South Carolina System; University of South Carolina Columbia
• 刊物名称: JOURNAL OF APPLIED PSYCHOLOGY
• ISSN/ISSBN: 0021-9010
• DOI: 10.1037/apl0000251
• 发表日期: 2018
• 页码: 37-53
• 关键词: intervention effects; statistical power; estimation precision; change; statistical models

摘要

Calls continue for randomized interventions in organizational settings. In many cases, however, practical constraints require researchers to use 2-wave randomized pretest-posttest control group designs. We discuss the importance of randomized trials for theory development with a focus on analytic options for 2-wave designs. Our discussion has implications for both designing studies and interpreting results. We review 23 published work and organizational health psychology intervention studies and find that a majority of studies featured a statistical model known to have low statistical power relative to other options. Furthermore, a majority of studies invoked terminology implying the direction of change without providing explicit statistical tests. To improve research practice, we detail statistical power differences in 3 commonly used statistical models and emphasize the distinction between (a) intervention effects and (b) the size and direction of change over time. We encourage researchers to provide inferential evidence for both types of information and show that only 1 of the 3 reviewed models provides information on the direction of change over time, but at a potential expense for statistical power to detect intervention effects. A reanalysis of data from a published work-family workplace intervention illustrates these nuances and supports recommendations for research practice. We conclude by providing recommendations.