Selective Inference for Hierarchical Clustering

• 成果类型: Article
• 署名作者: Gao, Lucy L.; Bien, Jacob; Witten, Daniela
• 署名单位: University of British Columbia; University of Southern California; University of Washington; University of Washington Seattle; University of Washington; University of Washington Seattle
• 刊物名称: JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION
• ISSN/ISSBN: 0162-1459
• DOI: 10.1080/01621459.2022.2116331
• 发表日期: 2024
• 页码: 332-342
• 关键词: statistical significance; single

摘要

Classical tests for a difference in means control the Type I error rate when the groups are defined a priori. However, when the groups are instead defined via clustering, then applying a classical test yields an extremely inflated Type I error rate. Notably, this problem persists even if two separate and independent datasets are used to define the groups and to test for a difference in their means. To address this problem, in this article, we propose a selective inference approach to test for a difference in means between two clusters. Our procedure controls the selective Type I error rate by accounting for the fact that the choice of null hypothesis was made based on the data. We describe how to efficiently compute exact p-values for clusters obtained using agglomerative hierarchical clustering with many commonly used linkages. We apply our method to simulated data and to single-cell RNA-sequencing data. for this article are available online.