Data-Driven Structured Policy Iteration for Homogeneous Distributed Systems

• 成果类型: Article
• 署名作者: Alemzadeh, Siavash; Talebi, Shahriar; Mesbahi, Mehran
• 署名单位: University of Washington; University of Washington Seattle; Microsoft; Harvard University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2024.3366038
• 发表日期: 2024
• 页码: 5979-5994
• 关键词: costs; Artificial neural networks; large-scale systems; Distributed feedback devices; decentralized control; Symmetric matrices; scalability; Data-driven policy iteration; networked control systems; Patterned monoids; structured control

摘要

Control of networked systems, comprised of interacting agents, is often achieved through modeling the underlying interactions. Constructing accurate models of such interactions-in the meantime-can become prohibitive in applications. Data-driven control methods avoid such complications by directly synthesizing a controller from the observed data. In this article, we propose an algorithm referred to as data-driven structured policy iteration (D2SPI), for synthesizing an efficient feedback mechanism that respects the sparsity pattern induced by the underlying interaction network. In particular, our algorithm uses temporary auxiliary communication links in order to enable the required information exchange on a (smaller) subnetwork during the learning phase-links that will be removed subsequently for the final distributed feedback synthesis. We then proceed to show that the learned policy results in a stabilizing structured policy for the entire network. Our analysis is then followed by showing the stability and convergence of the proposed distributed policies throughout the learning phase, exploiting a construct referred to as the Patterned monoid. The performance of data-driven structured policy iteration (D2SPI) is then demonstrated using representative simulation scenarios.