Nonlinear conjugate gradient methods: worst-case convergence rates via computer-assisted analyses

• 成果类型: Article
• 署名作者: Das Gupta, Shuvomoy; Freund, Robert M.; Sun, Xu Andy; Taylor, Adrien
• 署名单位: Massachusetts Institute of Technology (MIT); Massachusetts Institute of Technology (MIT); Centre National de la Recherche Scientifique (CNRS); Universite PSL; Inria
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-024-02127-7
• 发表日期: 2025
• 页码: 1-49
• 关键词: line search technique; GLOBAL CONVERGENCE; 1st-order methods; case performance; minimization; semidefinite; algorithm; descent

摘要

We propose a computer-assisted approach to the analysis of the worst-case convergence of nonlinear conjugate gradient methods (NCGMs). Those methods are known for their generally good empirical performances for large-scale optimization, while having relatively incomplete analyses. Using our computer-assisted approach, we establish novel complexity bounds for the Polak-Ribi & egrave;re-Polyak (PRP) and the Fletcher-Reeves (FR) NCGMs for smooth strongly convex minimization. In particular, we construct mathematical proofs that establish the first non-asymptotic convergence bound for FR (which is historically the first developed NCGM), and a much improved non-asymptotic convergence bound for PRP. Additionally, we provide simple adversarial examples on which these methods do not perform better than gradient descent with exact line search, leaving very little room for improvements on the same class of problems.