Scalable Reinforcement Learning for Multiagent Networked Systems

• 成果类型: Article
• 署名作者: Qu, Guannan; Wierman, Adam; Li, Na
• 署名单位: Carnegie Mellon University; California Institute of Technology; Harvard University
• 刊物名称: OPERATIONS RESEARCH
• ISSN/ISSBN: 0030-364X
• DOI: 10.1287/opre.2021.2226
• 发表日期: 2022
• 页码: 3601-3628
• 关键词: csma; complexity; average

摘要

We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find localized policies such that the (discounted) global reward is maximized. A fundamental challenge in this setting is that the state-action space size scales exponentially in the number of agents, rendering the problem intractable for large networks. In this paper, we propose a scalable actor critic (SAC) framework that exploits the network structure and finds a localized policy that is an O(rho(kappa+1))-approximation of a stationary point of the objective for some rho is an element of(0, 1), with complexity that scales with the local state-action space size of the largest kappa-hop neighborhood of the network. We illustrate our model and approach using examples fromwireless communication, epidemics, and traffic.