Goal scoring, coherent loss and applications to machine learning

• 成果类型: Article
• 署名作者: Yang, Wenzhuo; Sim, Melvyn; Xu, Huan
• 署名单位: National University of Singapore; National University of Singapore; University System of Georgia; Georgia Institute of Technology
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-019-01387-y
• 发表日期: 2020
• 页码: 103-140
• 关键词: Classification; optimization; probability

摘要

Motivated by the binary classification problem in machine learning, we study in this paper a class of decision problems where the decision maker has a list of goals, from which he aims to attain the maximal possible number of goals. In binary classification, this essentially means seeking a prediction rule to achieve the lowest probability of misclassification, and computationally it involves minimizing a (difficult) non-convex, 0-1 loss function. To address the intractability, previous methods consider minimizing thecumulative loss-the sum of convex surrogates of the 0-1 loss of each goal. We revisit this paradigm and develop instead anaxiomaticframework by proposing a set of salient properties on functions for goal scoring and then propose thecoherent lossapproach, which is a tractable upper-bound of the loss over theentire setof goals. We show that the proposed approach yields a strictly tighter approximation to the total loss (i.e., the number of missed goals) than any convex cumulative loss approach while preserving the convexity of the underlying optimization problem. Moreover, this approach, applied to for binary classification, also has a robustness interpretation which builds a connection to robust SVMs.