Consistent learning by composite proximal thresholding

• 成果类型: Article
• 署名作者: Combettes, Patrick L.; Salzo, Saverio; Villa, Silvia
• 署名单位: North Carolina State University; Istituto Italiano di Tecnologia - IIT; Polytechnic University of Milan
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-017-1133-8
• 发表日期: 2018
• 页码: 99-127
• 关键词: INVERSE PROBLEMS; signal recovery; minimization; CONVERGENCE; algorithms; regularization; sparsity; inexact; point

摘要

We investigate the modeling and the numerical solution of machine learning problems with prediction functions which are linear combinations of elements of a possibly infinite dictionary of functions. We propose a novel flexible composite regularization model, which makes it possible to incorporate various priors on the coefficients of the prediction function, including sparsity and hard constraints. We show that the estimators obtained by minimizing the regularized empirical risk are consistent in a statistical sense, and we design an error-tolerant composite proximal thresholding algorithm for computing such estimators. New results on the asymptotic behavior of the proximal forward-backward splitting method are derived and exploited to establish the convergence properties of the proposed algorithm. In particular, our method features a o(1 / m) convergence rate in objective values.