Adaptive Dynamic Programming for Optimal Control of Unknown LTI System via Interval Excitation

• 成果类型: Article
• 署名作者: Ma, Yong-Sheng; Sun, Jian; Xu, Yong; Cui, Shi-Sheng; Wu, Zheng-Guang
• 署名单位: Beijing Institute of Technology; Zhejiang University
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2025.3542328
• 发表日期: 2025
• 页码: 4896-4903
• 关键词: Optimal control; Heuristic algorithms; Linear systems; CONVERGENCE; memory; Approximation algorithms; dynamic programming; Adaptive systems; Adaptation models; training; Adaptive dynamic programming (ADP); optimal control; policy iteration (PI)

摘要

In this article, we investigate the optimal control problem for an unknown linear time-invariant system. To solve this problem, a novel composite policy iteration algorithm based on adaptive dynamic programming is developed to adaptively learn the optimal control policy from system data. The existing methods require the initial stabilizing control policy, the persistence of excitation (PE) condition and the data storage to ensure the algorithm convergence. Fundamentally different from them, these restrictions can be relaxed in the proposed method. Specifically, an adaptive parameter is elaborately designed to remove the requirement of the initial stabilizing control policy. Besides, an online data calculation scheme is proposed, which cannot only replace the stored historical data by online data, but also can relax the PE condition to the interval excitation condition. The simulation results demonstrate the efficacy of the proposed algorithm, and its superiority is also demonstrated by comparing it with existing algorithms.