Off-Policy Reinforcement Learning for H∞ Control of Linear Discrete-Time Systems With Network-Induced Dropouts

• 成果类型: Article
• 署名作者: Jiang, Yi; Yang, Tao; Gao, Weinan; Wu, Jin; Chai, Tianyou; Lewis, Frank L.
• 署名单位: Huazhong University of Science & Technology; Huazhong University of Science & Technology; Northeastern University - China; Northeastern University - China; University of Science & Technology Beijing; University of Texas System; University of Texas Arlington
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3582529
• 发表日期: 2025
• 关键词: Zero-sum games; DESIGN

摘要

This article studies an adaptive discrete-time linear H-infinity control problem with networked induced dropouts. First, such a problem is formulated as a zero-sum game problem, and it is shown that the formulated problem can be solved via an optimal feedback control policy and a worst disturbance policy. These policies result from one positive-definite solution to a modified game algebraic Riccati equation (MGARE). Then, the solvability of the MGARE, the stochastic asymptotical stability, and the disturbance attenuation level of the closed-loop system are rigorously analyzed. To obtain such solution to the MGARE, two model-based reinforcement learning (RL) algorithms, namely, policy iteration (PI) and value iteration (VI) algorithms, are proposed, and their corresponding convergence analysis are given as well. Based on model-based RL algorithms, two data-driven RL algorithms, namely, data-driven PI and VI algorithms, are designed by directly using the data transmitted via communication networks in a model-free sense, in which the optimal control policy and the worst disturbance policy are, thus, obtained iteratively. Furthermore, a data-driven computation algorithm for drawing the feasible area of such MGARE approximately is designed. Finally, simulation examples are given to show the effectiveness of the proposed approaches.