Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments

• 成果类型: Article
• 署名作者: Muralidharan, Karthik; Romero, Mauricio; Wuethrich, Kaspar
• 署名单位: University of California System; University of California San Diego; National Bureau of Economic Research; Instituto Tecnologico Autonomo de Mexico; University of Michigan System; University of Michigan; Leibniz Association; Ifo Institut
• 刊物名称: REVIEW OF ECONOMICS AND STATISTICS
• ISSN/ISSBN: 0034-6535
• DOI: 10.1162/rest_a_01317
• 发表日期: 2025-05
• 页码: 589-604
• 关键词: field experiment; confidence-intervals; education; tests; BIAS

摘要

Factorial designs are widely used to study multiple treatments in one experiment. Although t-tests using a fully saturated long model provide valid inferences, short model t-tests (that ignore interactions) yield higher power if interactions are zero, but incorrect inferences otherwise. Of 27 factorial experiments published in top-five journals (2007-2017), nineteen use the short model. After including interactions, more than half of their results lose significance. Based on recent econometric advances, we show that power improvements over the long model are possible. We provide practical guidance for the design of new experiments and the analysis of completed experiments.