A family of second-order methods for convex -regularized optimization

• 成果类型: Article
• 署名作者: Byrd, Richard H.; Chin, Gillian M.; Nocedal, Jorge; Oztoprak, Figen
• 署名单位: University of Colorado System; University of Colorado Boulder; Northwestern University; Istanbul Bilgi University
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-015-0965-3
• 发表日期: 2016
• 页码: 435-467
• 关键词: thresholding algorithm; newton; shrinkage; strategy; online

摘要

This paper is concerned with the minimization of an objective that is the sum of a convex function f and an regularization term. Our interest is in active-set methods that incorporate second-order information about the function f to accelerate convergence. We describe a semismooth Newton framework that can be used to generate a variety of second-order methods, including block active set methods, orthant-based methods and a second-order iterative soft-thresholding method. The paper proposes a new active set method that performs multiple changes in the active manifold estimate at every iteration, and employs a mechanism for correcting these estimates, when needed. This corrective mechanism is also evaluated in an orthant-based method. Numerical tests comparing the performance of three active set methods are presented.