Probability estimation for large-margin classifiers

• 成果类型: Article
• 署名作者: Wang, Junhui; Shen, Xiaotong; Liu, Yufeng
• 署名单位: University of Minnesota System; University of Minnesota Twin Cities; University of North Carolina; University of North Carolina Chapel Hill
• 刊物名称: BIOMETRIKA
• ISSN/ISSBN: 0006-3444
• DOI: 10.1093/biomet/asm077
• 发表日期: 2008
• 页码: 149167
• 关键词: Support vector machines; CLASSIFICATION; prediction; selection; cancer; error

摘要

Large margin classifiers have proven to be effective in delivering high predictive accuracy, particularly those focusing on the decision boundaries and bypassing the requirement of estimating the class probability given input for discrimination. As a result, these classifiers may not directly yield an estimated class probability, which is of interest itself. To overcome this difficulty, this article proposes a novel method for estimating the class probability through sequential classifications, by using features of interval estimation of large-margin classifiers. The method uses sequential classifications to bracket the class probability to yield an estimate up to the desired level of accuracy. The method is implemented for support vector machines and psi-learning, in addition to an estimated Kullback-Leibler loss for tuning. A solution path of the method is derived for support vector machines to reduce further its computational cost. Theoretical and numerical analyses indicate that the method is highly competitive against alternatives, especially when the dimension of the input greatly exceeds the sample size. Finally, an application to leukaemia data is described.