Safe Reinforcement Learning Using Robust MPC

• 成果类型: Article
• 署名作者: Zanon, Mario; Gros, Sebastien
• 署名单位: IMT School for Advanced Studies Lucca; Norwegian University of Science & Technology (NTNU)
• 刊物名称: IEEE TRANSACTIONS ON AUTOMATIC CONTROL
• ISSN/ISSBN: 0018-9286
• DOI: 10.1109/TAC.2020.3024161
• 发表日期: 2021
• 页码: 3638-3652
• 关键词: safety; Robustness; data models; Numerical models; uncertainty; stability analysis; computational modeling; reinforcement learning (RL); robust model predictive control (MPC); safe policies

摘要

Reinforcement learning (RL) has recently impressed the world with stunning results in various applications. While the potential of RL is now well established, many critical aspects still need to be tackled, including safety and stability issues. These issues, while secondary for the RL community, are central to the control community that has been widely investigating them. Model predictive control (MPC) is one of the most successful control techniques because, among others, of its ability to provide such guarantees even for uncertain constrained systems. Since MPC is an optimization-based technique, optimality has also often been claimed. Unfortunately, the performance of MPC is highly dependent on the accuracy of the model used for predictions. In this article, we propose to combine RL and MPC in order to exploit the advantages of both, and therefore, obtain a controller that is optimal and safe. We illustrate the results with two numerical examples in simulations.