Robust Policy Iteration for Continuous-Time Linear Quadratic Regulation

• 成果类型: Article
• 署名作者: Pang, Bo; Bian, Tao; Jiang, Zhong-Ping
• 署名单位: New York University; New York University Tandon School of Engineering; Bank of America Corporation
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2021.3085510
• 发表日期: 2022
• 页码: 504-511
• 关键词: robustness; Biological system modeling; optimal control; Numerical stability; Heuristic algorithms; Approximation algorithms; Symmetric matrices; Adaptive dynamic programming; Adaptive optimal control; Data-driven control; policy iteration; Reinforcement Learning; Robustness

摘要

This article studies the robustness of policy iteration in the context of continuous-time infinite-horizon linear quadratic regulator (LQR) problem. It is shown that Kleinman's policy iteration algorithm is small-disturbance input-to-state stable, a property that is stronger than Sontag's local input-to-state stability but weaker than global input-to-state stability. More precisely, whenever the error in each iteration is bounded and small, the solutions of the policy iteration algorithm are also bounded and enter a small neighborhood of the optimal solution of the LQR problem. Based on this result, an off-policy data-driven policy iteration algorithm for the LQR problem is shown to be robust when the system dynamics are subject to small additive unknown bounded disturbances. The theoretical results are validated by a numerical example.