Decentralized Online Convex Optimization With Feedback Delays

• 成果类型: Article
• 署名作者: Cao, Xuanyu; Basar, Tamer
• 署名单位: University of Illinois System; University of Illinois Urbana-Champaign
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2021.3092562
• 发表日期: 2022
• 页码: 2889-2904
• 关键词: Delays; optimization; Heuristic algorithms; Benchmark testing; Convex functions; sensors; measurement; Bandit feedback; decentralized optimization; feedback delays; full information; online convex optimization (OCO)

摘要

In online decision making, feedback delays often arise due to the latency caused by computation and communication in practical systems. In this article, we study decentralized online convex optimization over a multiagent network in the presence of feedback delays. Each agent is associated with a time-varying local loss function, which is revealed to the agent sequentially with delays. The goal of every agent is to minimize the accumulated total loss function (the sum of the local loss functions of all agents) by choosing appropriate local actions. We first develop an online decentralized gradient descent algorithm for the delayed full information case, in which the delayed local loss function is fully disclosed to each agent. The (static) regret of each agent is upper bounded in terms of the delays for both convex and strongly convex loss functions. We show that, as long as the average delay among all agents is sublinear, so is the regret of every agent. Further, we extend the algorithm and analysis to the scenario of bandit feedback, where only the values of the delayed loss functions at two random points close to the local action are revealed. We demonstrate that the two-point bandit feedback does not entail regret degradation in order sense compared to the full information scenario. Finally, numerical results are presented to corroborate the efficacy of the proposed algorithms.