Distributed Optimization With Personalization: A Flexible Algorithmic Framework

• 成果类型: Article
• 署名作者: Huang, Yan; Xu, Jinming; Meng, Wenchao; Wai, Hoi-To; Chai, Li
• 署名单位: Zhejiang University; Chinese University of Hong Kong
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2024.3381889
• 发表日期: 2024
• 页码: 6715-6730
• 关键词: Peer-to-peer computing; CONVERGENCE; Stochastic processes; linear programming; optimization; vectors; STANDARDS; distributed optimization; PERSONALIZATION; uncoordinated update

摘要

We consider a distributed personalized optimization problem over a network of nodes whose cost functions depend on a decision variable consisting of two parts: global (shared) part and local (node-specific) part. This problem structure arises in several important scenarios, where the global part captures the common feature among nodes and the local part represents the personalized feature. To solve this problem, we develop a flexible algorithmic framework employing personalized distributed stochastic gradient tracking method, where each node updates variables locally and communicates the shared part with its neighbors for coordination in a stochastic way. Different from existing works, the proposed framework provides great flexibility in designing algorithms with uncoordinated stepsizes for updating local and global parts over changing topologies. Most importantly, it can eliminate the effect of heterogeneity among nodes thanks to the gradient tracking scheme. Leveraging a properly designed Lyapunov function, we prove the convergence of the proposed algorithm for both (strongly) convex and nonconvex objective functions with cross-block Lipschitz gradients. The obtained rate results show clear dependence of the convergence performance on topology, heterogeneity of stepsize, and properties of objective functions. Numerical examples are provided to demonstrate the effectiveness of the proposed algorithm accounting for node-specific heterogeneity.