THE IMPOSSIBILITY REGION FOR DETECTING SPARSE MIXTURES USING THE HIGHER CRITICISM

• 成果类型: Article
• 署名作者: Donoho, David l.; Kipnis, A. lon
• 署名单位: Stanford University; Reichman University
• 刊物名称: ANNALS OF APPLIED PROBABILITY
• ISSN/ISSBN: 1050-5164
• DOI: 10.1214/24-AAP2086
• 发表日期: 2024
• 页码: 4921-4939
• 关键词: feature-selection; tests; rare

摘要

Consider a multiple hypothesis testing setting involving rare/weak effects: relatively few tests, out of possibly many, deviate from their null hypothesis behavior. Summarizing the significance of each test by a p-value, we construct a global test against the joint null using the higher criticism (HC) statistics of these p-values. We calibrate the rare/weak model using parameters controlling the asymptotic distribution of nonnull p-values near zero. We derive a region in the parameter space where the HC test is asymptotically powerless. Our derivation involves very different tools than previously used to show powerlessness of HC, relying on properties of the empirical processes underlying HC. In particular, our result applies to situations where HC is not asymptotically optimal, or when the asymptotically detectable region of the parameter space is unknown.