Unifying mirror descent and dual averaging

• 成果类型: Article
• 署名作者: Juditsky, Anatoli; Kwon, Joon; Moulines, Eric
• 署名单位: Communaute Universite Grenoble Alpes; Universite Grenoble Alpes (UGA); AgroParisTech; INRAE; Universite Paris Saclay; Institut Polytechnique de Paris; Ecole Polytechnique
• 刊物名称: MATHEMATICAL PROGRAMMING
• ISSN/ISSBN: 0025-5610
• DOI: 10.1007/s10107-022-01850-3
• 发表日期: 2023
• 页码: 793-830
• 关键词: variational-inequalities; minimization; algorithms

摘要

We introduce and analyze a new family of first-order optimization algorithms which generalizes and unifies both mirror descent and dual averaging. Within the framework of this family, we define new algorithms for constrained optimization that combines the advantages of mirror descent and dual averaging. Our preliminary simulation study shows that these new algorithms significantly outperform available methods in some situations.