On the Sample Complexity of Decentralized Linear Quadratic Regulator With Partially Nested Information Structure

• 成果类型: Article
• 署名作者: Ye, Lintao; Zhu, Hao; Gupta, Vijay
• 署名单位: Huazhong University of Science & Technology; Huazhong University of Science & Technology; University of Texas System; University of Texas Austin; Purdue University System; Purdue University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2022.3215940
• 发表日期: 2023
• 页码: 4841-4856
• 关键词: Decentralized control; large-scale systems; optimal control; Reinforcement Learning; System identification; Statistical learning

摘要

In this article, we study the problem of control policy design for decentralized state-feedback linear quadratic control with a partially nested information structure, when the system model is unknown. We propose a model-based learning solution, which consists of two steps. First, we estimate the unknown system model from a single system trajectory of finite length, using least squares estimation. Next, based on the estimated system model, we design a decentralized control policy that satisfies the desired information structure. We show that the suboptimality gap between our control policy and the optimal decentralized control policy (designed using accurate knowledge of the system model) scales linearly with the estimation error of the system model. Using this result, we provide an end-to-end sample complexity result for learning decentralized controllers for a linear quadratic control problem with a partially nested information structure.