A Note on State Parameterizations in Output Feedback Reinforcement Learning Control of Linear Systems

• 成果类型: Article
• 署名作者: Rizvi, Syed Ali Asad; Lin, Zongli
• 署名单位: Tennessee Technological University; University of Virginia
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2022.3228969
• 发表日期: 2023
• 页码: 6200-6207
• 关键词: Output feedback; State feedback; CONVERGENCE; observers; Q-learning; regulators; observability; Adaptive dynamic programming; optimal control; output feedback control; reinforcement learning (RL); state parameterization

摘要

This note presents an analysis of the state parameterizations used in output feedback reinforcement learning (RL) control. Output feedback algorithms based on state parameterization involve additional conditions on the state parameterization beyond the standard conditions on the system matrices for their convergence to the optimal solution. It is shown that the state parameterization matrix needs to be of full row rank to guarantee the convergence of the output feedback RL algorithms. We present conditions in terms of the system matrices and the user-defined observer dynamics that ensure full row rank of the state parameterization matrix.