Developing Lagrangian-Based Methods for Nonsmooth Nonconvex Optimization

• 成果类型: Article; Early Access
• 署名作者: Xiao, Nachuan; Ding, Kuangyu; Hu, Xiaoyin; Toh, Kim-Chuan
• 署名单位: The Chinese University of Hong Kong, Shenzhen; Purdue University System; Purdue University; Shenzhen University; National University of Singapore; National University of Singapore
• 刊物名称: MATHEMATICS OF OPERATIONS RESEARCH
• ISSN/ISSBN: 0364-765X; 1526-5471
• DOI: 10.1287/moor.2024.0479
• 发表日期: 2026-01-08
• 关键词: nonsmooth optimization; constrained optimization; Lagrangian-based methods; stochastic subgradient method; Deep learning; convex-optimization; constraints; composite

摘要

In this paper, we consider the minimization of a nonsmooth nonconvex objective function f (x) over a closed convex subset X of Rn, with additional nonsmooth nonconvex constraints c(x) = 0. We develop a unified framework for developing Lagrangian-based methods, which takes a single-step update to the primal variables by some subgradient methods in each iteration. These subgradient methods are embedded into our framework in the sense that they are incorporated as black-box updates to the primal variables. We prove that our proposed framework inherits the global convergence guarantees from these embedded subgradient methods under mild conditions. In addition, we show that our framework can be extended to solve constrained optimization problems with expectation constraints. Based on the proposed framework, we show that a wide range of existing stochastic subgradient methods, including proximal stochastic subgradient descent (SGD), proximal momentum SGD, and proximal adaptive moment estimation method (ADAM), can be embedded into Lagrangian-based methods. Preliminary numerical experiments on deep learning tasks illustrate that our proposed framework yields efficient variants of Lagrangian-based methods with convergence guarantees for nonsmooth nonconvex constrained optimization problems.