ADAPTIVE NOVELTY DETECTION WITH FALSE DISCOVERY RATE GUARANTEE

• 成果类型: Article
• 署名作者: Marandon, Ariane; Lei, Lihua; Mary, David; Roquain, Etienne
• 署名单位: Sorbonne Universite; Universite Paris Cite; Centre National de la Recherche Scientifique (CNRS); Stanford University; Universite Cote d'Azur; Observatoire de la Cote d'Azur
• 刊物名称: ANNALS OF STATISTICS
• ISSN/ISSBN: 0090-5364
• DOI: 10.1214/23-AOS2338
• 发表日期: 2024
• 页码: 157-183
• 关键词: Empirical Bayes; inference; selection

摘要

This paper studies the semisupervised novelty detection problem where a set of typical measurements is available to the researcher. Motivated by recent advances in multiple testing and conformal inference, we propose AdaDetect, a flexible method that is able to wrap around any probabilistic classification algorithm and control the false discovery rate (FDR) on detected novelties in finite samples without any distributional assumption other than exchangeability. In contrast to classical FDR-controlling procedures that are often committed to a pre-specified p-value function, AdaDetect learns the transformation in a data-adaptive manner to focus the power on the directions that distinguish between inliers and outliers. Inspired by the multiple testing literature, we further propose variants of AdaDetect that are adaptive to the proportion of nulls while maintaining the finite-sample FDR control. The methods are illustrated on synthetic datasets and real-world datasets, including an application in astrophysics.