Rank-transformed subsampling: inference for multiple data splitting and exchangeable p-values

• 成果类型: Article
• 署名作者: Guo, F. Richard; Shah, Rajen D.
• 署名单位: University of Michigan System; University of Michigan; University of Cambridge
• 刊物名称: JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY
• ISSN/ISSBN: 1369-7412
• DOI: 10.1093/jrsssb/qkae091
• 发表日期: 2024
• 页码: 256-286
• 关键词: of-fit test; statistical significance; sample; tests; combination; parameters; bootstrap; CHOICE; error

摘要

Many testing problems are readily amenable to randomized tests, such as those employing data splitting. However, despite their usefulness in principle, randomized tests have obvious drawbacks. Firstly, two analyses of the same dataset may lead to different results. Secondly, the test typically loses power because it does not fully utilize the entire sample. As a remedy to these drawbacks, we study how to combine the test statistics or p-values resulting from multiple random realizations, such as through random data splits. We develop rank-transformed subsampling as a general method for delivering large-sample inference about the combined statistic or p-value under mild assumptions. We apply our methodology to a wide range of problems, including testing unimodality in high-dimensional data, testing goodness-of-fit of parametric quantile regression models, testing no direct effect in a sequentially randomized trial and calibrating cross-fit double machine learning confidence intervals. In contrast to existing p-value aggregation schemes that can be highly conservative, our method enjoys Type I error control that asymptotically approaches the nominal level. Moreover, compared to using the ordinary subsampling, we show that our rank transform can remove the first-order bias in approximating the null under alternatives and greatly improve power.